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Abstract 

Home schooling is a subject of great fascination, but little solid knowledge. Despite its 
importance, it has received less research attention than some other recent changes in the 
educational system, such as the growth of charter schools. It could be argued that home schooling 
may have a much larger impact on educational system, both in the short and long run. This report 
uses the 1994 October CPS, and the National Household Education Survey of 1996 and 1999 to 
examine popular characterizations of the home school population. The article assembles evidence 
from several sources to confirm that home schooling is growing. It finds home-schooled children 
more likely to^be middle income, white, from larger families, and from two-parent families with 
one parent not working. While some authors have described a division between religiously- 
motivated and academically-motivated home schoolers, this research finds more support for a 
divide based on attitude towards regular schools. 



The Impact of Home Schooling 

Home schooling is a subject of great fascination, but little solid knowledge. Compared with other recent changes in 
the educational system, such as the growth of charter schools, home schooling has received relatively little attention 
(Archer 2000). (Note 1) It could be argued, however, that home schooling could have a much larger impact on 
educational system, both in the short and long run. This is because home schooling seems to be taking place on a 
larger scale than many other educational innovations (Lines 1999, Bieleck 2001), because home schooling may have a 
greater immediate impact on educational practices in existing schools (Hill 2000, Lines 2000b), and because home 
schooling has brought new institutional forms into being that have the potential to grow over the longer term (Trotter 
2001 ). 

Scale 




Although other institutional innovations in the educational system have grown in recent years, home schooling is 
probably the largest change in the sheer number of students involved. 



J . 



4 



best COPT 




Home schooling directly comprises a larger student population than voucher school programs— at least those that 
include private schools, that enroll only a few thousand students in a few cities (see Gardner 2000). Home schooling 
also involves a larger population than charter schools. According to estimates from organizations involved with 
charter schools, the student population in the fall of 2000 was just over 500,000. (Center for Education Reform, 2001). 
Even conservative estimates of the number of home schoolers put their numbers at that level or above (Lines 1999). 

Organizational changes 



Charter schools and voucher systems provide competitive challenges to traditional public schools, and as such, 
provide a direct incentive to adopt innovations and match the performance of other schools. However, the main 
outlines of current schooling practice have thus far remained intact. The challenge of home schooling, by contrast, is 
more profound. Home schooling is a more radical departure from education as it is currently practiced, it affects rnore 
schools, and it has the potential to force numerous adjustments to current curricular practices. 



Public schools in many jurisdictions have already begun to provide services of various types to home schoolers. Laws 
in at least seven states permit home schooled students to participate in sports, music and other extracurricular 
activities in regular schools (Farris 1997). In Florida and Iowa, schools also allow home schoolers to take individual 
courses. 

New Institutions 



Perhaps the largest impact of home schoolers has been the concomitant entry of new educational organizations into 
the field. Many private organizations and enterprises have entered the K-12 distance education field with their sights 
set on home schoolers as a primary audience (Hill 2000). The State of Florida has developed an extensive set of 
courses that can be taken over the Internet for high school credit by home schoolers and others who choose to use this 
resource, and Illinois is developing a similar program (Carothers 2000, Trotter 2001). Meanwhile several for-profit 
ventures have entered the field, offering courses and, in one case, accredited diplomas over the Internet (Trotter 1999 
Walsh 2001). 

If home schooling continues to grow, demand will grow for the types of services that are starting to be offered by 
public schools and distance education providers. A result will be pressure on schools to design school curricula that 
allow students and parents to pick and choose what they like. According to some observers, another result will be the 
creation of new schools and school-like institutions built around the common needs and concerns of home-schooling 
families (Hill 2000) and the growth of public school programs designed specifically for home schoolers (Lines 
2000b). ^ 



Despite these broad impacts there have been few attempts to examine the characteristics of home schoolers and their 
households in the U.S. Many studies that have been conducted have relied on highly selective samples (Rudner 1999, 
Weiner & Weiner 1999) or have examined selective issues without giving a thorough overview of the home-schooled 
population (Smith & Sikkink 1999, Weiner 2000a, Weiner 2000b, Lines 2000b). The two exceptions are reports by 
Lines (1999) and the National Center for Education Statistics (Bielick 2001) who provide estimates of the home- 
school population. Lines conducted a careful analysis state education agency records of registered home schoolers, 
adjusting for probable levels of non-registered home schooling. She estimated that there were 690,000 home-schooled 
children in 1995. The National Center for Education Statistics report analyzed the results of the 1999 National 
Household Education Survey, which is also one of the data sets also analyzed in this article. They produced basic 
tabulations of the characteristics of home-schoolers, including grade equivalent, race, sex, family characteristics, 
participation in public schools and reasons for home schooling. They found 850,000 home schooled children in the 
United States. (This is a larger figure than the one reported here, because they decided to include 5-year-olds in the 
count of home-schooled children, while this report includes only those age 6 to 1 7.) Prior to these resports, there was 
also an especially careful attempt by researchers associated with the U.S. Department of Education to reconcile results 
from two major national surveys measuring the home school population (Henke et al. 2000). Unfortunately, the 
authors of that publication did not have more recent data available to them. 



This article adds to the current knowledge on the subject by looking simultaneously at three national datasets on home 
schooling. The report takes a closer look at the characteristics of home schoolers and tests for the significance of 
differences between home-schooled children and others. It examines trends and compositional changes in the home- 
schooled population. It examines their geographic location and potential for growth. Finally, it examines whether 
there are identifiable groups of home schoolers with different reasons for pursuing home-schooling, as has been 
posited by many observers. 

The article proceeds as follows. It starts with a discussion of the data sources used in the analysis. Next the number of 
home schoolers and the rate of growth is estimated from various data sets. The subsequent section examines 
characteristics of home schooled children and their families, with a focus on those characteristics most relevant for 
gauging trends in home schooling. Finally, there is a discussion of some of the implications of home schooling for 
regular schools and a brief conclusion. 



Data on Home Schooling 



The data for this project include the 1994 October Current Population Survey (CPS) (U.S. Census Bureau 2000) and 
the National Household Education Surveys (NHES) of 1996 and 1999 (Nolin et al. 2000). All three are national 
household surveys of high quality. The CPS relies on a combination of in-person and telephone interviewing of a 
large sample (approximately 60,000 households) of the U.S. population. 1 use 24,829 CPS cases where subjects were 
age 6 to 17. In October of each year, a supplement on school enrollment of children and adults is administered in all 
CPS households. The content of this supplement varies slightly from year to year, and in 1 994 questions on home 
schooling were added to the main enrollment questions in the supplement for children. The questions differed 
according to the response to the initial question on school enrollment. 

If it was reported that a child was not currently enrolled in school, the child or proxy was asked: 

’’Were you/Was ... being schooled primarily at home?” 



If the child was currently in school the question was: 

’’Are you/Is ... attending (1) a regular day school, (2) boarding school, (3) schooled primarily at home 
by someone paid by the school, (4) schooled primarily at home by a parent or other person paid or 
chosen by a parent, (5) someplace else." 

The number choosing answer (3) was relatively small, and for the purposes of this research, responses (3) and (4) 
were both counted as "home schooling.” 

The NHES surveys are nationally-representative telephone surveys administered by the National Center for Education 
Statistics. The two most recent surveys, in 1996 and 1999 have included questions on home schooling. The number of 
children 6 to 1 7 was 1 6,257 in 1 996 and 1 0,7 1 8 in 1 999. 

In both years, the same question was asked of all children: 

"Some parents decide to educate their children at home rather than sending them to school. Is ... 
being schooled at home?” 

The datasets also provide several types of information on characteristics of home schoolers and their families. All 
provide race, Hispanic ethnicity, age, and sex of children. They also provide information on the household: number of 
adults in the household, their education, labor force participation and household income. In both the CPS and NHES, 
income was given in ranges. For regression analyses, these were recoded to the midpoints and differenced from the 
mean. CPS provided state of residence, metropolitan status and urban/rural location. Although it is traditional to use 
Census-defined regions for analyses, it was felt that home schooling may not be following traditional patterns. Frey 
(2000) developed a regional taxonomy that reflects the major migration patterns of recent years, and these are 
probably more closely related to the types of social trends that would affect home-schooling decisions. The states 
were recoded to regions following this migration taxonomy. An urban-rural division was developed from 
metropolitan and urban/rural variables in CPS. (Note 2) In both 1996 and 1999, the NHES asked parents of home 
schoolers about their motivations for teaching their children at home. Respondents were asked to select reasons from 
a list of 16. 



All analyses in this article use weighted data, adjusted to reflect an assumed design effect of 2.0, except that the 
standard errors associated with the total number of home schoolers were estimated using the Taylor-series 
linearization method available in the SAS statistical package. Specific types of analysis are described as they appear 
in the following discussion. 



Extent and Growth of Home Schooling 

Table 1 shows the number of home schooled children age 6 to 17 estimated from these data sources. Taken at face 
value, they show a growth from 360,000 in 1994 to 790,000 in 1999. By 1999, then, around 1.7 percent of children in 
the 6 to 17 age range were schooled at home. A 95 percent confidence interval for the 1999 figure goes from 670,000 
to 910,000. Even at the high end of the range, the home-school population is under 1 million and less then 2 percent 
of all children 6-17. 





Table 1 




Estimates of the Number of U.S. Children Schooled at Home: 
Current Population Survey & National Household Education Surveys 




Estimate 


Standard error 


CPS 1994 


356,000 


40,000 



6 



NHES 1996 


636,000 


54,000 


NHES 1999 


791,000 


62,000 



Under-reporting 

Because home schooling has become legal in most states only recently, and because regulations are sometimes 
cumbersome, there are a number of home-schoolers who have not reported their status to the state or local educational 
authorities, and would presumably be reluctant to report their status to interviewers. At the same time, other 
households may claim they are "home schooling" when they keep children away from school for other reasons or 
when they instruct their children while also sending them to school. Lines (1 999) produced a reasonable estimate of 
home-schooling by using reports from state education departments in conjunction with estimates of reporting rates 
from a survey by Ray (1997). It is possible to similarly check the CPS estimates against state agency reports state-by- 
state. 

I examined the 10 states with the highest and lowest reporting rates for which Lines was able to get state education 
department figures. CPS estimates were slightly lower than the number from state agencies in both cases. (Note 3) If 
Ray's estimates of reporting rates are reliable, therefore, in states where few home-schoolers reported to authorities, 
few reported to interviewers. Using a few simplifying assumptions, I calculated an "adjusted" number of home 
schoolers of 750,000 in 1994. (Note 4) If we assume 8 percent annual growth in home schooling, the NHES estimate 
from 1999 would be about 25% too low, and the actual number of home schoolers could be close to 1.1 million. 
However, this estimate depends critically on the validity of Ray's estimates of non-reporting (see discussion in Lines 
1 999). Until there is better evidence on the true rate of reporting, the unadjusted NHES figures are clearly the best 
available estimates. 

Growth in home schooling 



Unfortunately, the point estimates from these data cannot be used directly to make such inferences. The 1994 CPS 
estimate of 360,000 is not much more than half the size of the 1996 NHES estimate of 640,000. This difference is 
statistically significant, but is too large to be explained by growth in the home-school population. Hemke et ah (2000), 
noted that the gap is implausibly large, but were unable to pinpoint an explanation. A likely reason for the discrepancy 
is the difference in question wording between CPS and NHES. In the CPS, the form of the home schooling question 
depended on the previous answer to the question on school enrollment. If a household reported children were 
attending school, they were not asked directly about home schooling, but had to choose it from a list. That this results 
in a lower response is evident from the extremely low rate of home schooling observed in the subset of CPS 
respondents who responded affirmatively to the enrollment question. In the CPS, only 190,000 children were reported 
as in school, but also home schooled. In the 1 996 NHES, 450,000 children were reported this way. By contrast, 
people who initially indicated non-enrollment faced similar yes/no questions on home schooling in both surveys. 

They were much closer in number — 170,000 home schoolers in CPS and 190,000 in the 1996 NHES. 

The 1 999 NHES data seem also to show growth in home schooling. However, the growth is not quite statistically 
significant from 1 996, given the sample size (the p-value of the 1 996 to 1 999 difference is between .05 and . 1 0). 

Since the two NHES surveys are nearly identical in content and methodology, the trend based on these two data 
points provide the best estimate of growth, but the range is wide. A 95 percent confidence interval provides a range 
from 3 percent annual decline to 15 percent annual growth. 



At the first level of analysis, therefore, we can't say a lot about the growth of the home schooling population. We can, 
however, refute some of the grander claims that have been made by advocates. The number of home schooled 
children was well under 1 million in 1999, and the growth rate from 1996 to 1999 was unlikely to have exceeded 15 
percent per year. 

More evidence on growth 

The NHES data are insuficient to show growth in a statistical sense. However, if we can bring additional evidence to 
bear, we can increase our confidence that growth is actually taking place. One way to get additional evidence on 
trends in home schooling is to examine trends in reports of school non-enrollment. For children in the prime school- 
enrollment ages 7-9 and 10-13, published estimates show non-enrollment remained consistently at or below 1 percent 
from the mid 1950s to the early 1990s. From 1995 to 1999, however, non-enrollment exceeded 1 percent 4 out of 5 
years (Jamieson et al. 2001). An increase in the non-enrolled population is not the same as an increase in home 
schooling, but there is overlap. In the 7 to 14 age range, just under one-half of non-enrolled students were home 
schooled, according to tabulations from the 1994 CPS, and there is a correlation of around 0.5 between home- 
schooling and non-enrollment across states. A regression analysis of non-enrollment across years, using CPS data for 
1989 to 1999 shows a significant upward trend (data not shown — available from author on request). This confirms 
that the observed increase in recent years is not attributable to sampling error. 

A group that is especially likely to be home schooled consists of two-adult families with one not working (as will be 



shown below). In this group, 60 percent of non-enrolled children are home schooled. The regression of non- 
enrollment on years shows an equally large and significant coefficient for this group as it does for all school-aged 
children. 

In sum, evidence on non-enrollment reinforces the direct evidence available from the two NHES surveys: there seems 
to be an upward trend in home schooling. Other evidence might also be interpreted as supporting this conclusion, 
including demographic characteristics and geographic location. These are explored next. 



Characteristics of Home-Schooled Children 



To better understand trends in home schooling it is helpful to know what similarities and differences exist between 
home-schooled children and those in regular school. If home schoolers are currently limited to a portion of the 
population with distinct characteristics it is possible that the phenomenon will be self-contained. On the other hand, if 
those characteristics are becoming more prevalent in the population, then home schooling might grow along with the 
group in which ifs found. 



Home schoolers are like their peers in many respects. Table 2 shows how they compare, using data from all three 
surveys under consideration. Home schoolers are not especially likely to be young or old. They are about as likely to 
be of one sex or the other, with perhaps a slightly greater percentage female. In some ways, however, home-schoolers 
do stand out. Home schooled children are more likely to be non-Hispanic White, they are likely to live in households 
headed by a married couple with moderate to high levels of education and income. They are more likely to live in 
households with three or more children and they are likely to live in a household with an adult not in the labor force. 
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Table 2 

Characteristics of Home-Schooled Children and their Families 
Current Population Survey & National Household Education Surveys 




1994 


1996 


1999 


Home 

School 


Regular 

School 


Home 

School 


Regular 

School 


Home 

School 


Regular 

School 


Age 




6-7 


24.0 


17.2 


11.7 


17.4 


13.8 


17.8 


8-10 


30.6 


25.6 


25.9 


25.6 


26.1 


25.0 


11-14 


27.8 


33.8 


34.0 


33.1 


31.7 


32.4 


15-17 


17.5 


23.4 


28.5 


24.0 


28.4 


24.9 


Sex 




Male 


46.8 


51.1 


42.2 


51.5 


46.2 


50.9 


Female 


53.2 


48.9 


57.8 


48.5 


53.8 


49.1 


Number of children 




One child 


15.2 


20.6 


18.9 


21.2 


16.3 


21.4 


Two children 


20.9 


39.4 


25.8 


39.4 


29.8 


38.3 


Three or more 


63.9 


40.1 


55.2 


39.4 


53.9 


40.4 


Race, ethnicity 




White 


91.9 


67.6 


86.8 


67.7 


75.8 


64.8 


Black 


2.8 


15.9 


2.2 


15.6 


8.8 


16.1 


Hispanic 


4.4 


12.8 


8.0 


12.5 


9.1 


13.9 


Other 


0.8 


3.7 


3.1 


4.2 


6.2 


5.2 


Family structure 




Single parent 


11.3 


29.9 


20.8 


30.8 


20.6 


34.5 


Two parent 


88.7 


70.1 


79.2 


69.2 


79.4 


65.6 


Non-working parent 




Parents work 


34.0 


68.1 


41.3 


72.0 


38.8 


74.0 


Non-working parent 


66.0 


31.9 


58.7 


28.0 


61.2 


26.0 



o 



BEST 



Family income 






Up to 14,999 


18.8 


23.2 


21.1 


21.1 


12.3 


18.6 


15,000 to 29,999 


14.9 


20.4 


26.9 


22.6 


25.7 


21.3 


30,000 to 49,999 


40.4 


26.5 


29.1 


25.5 


24.8 


23.7 


50,000 or more 


25.9 


29.9 


22.9 


30.7 


37.1 


36.4 


Mother's education 




Less than h.s. 


8.8 


17.7 


14.2 


16.4 


5.3 


16.4 


High school 


31.2 


35.4 


23.6 


33.7 


28.9 


29.2 


Some college 


37.9 


28.9 


40.5 


28.3 


34.3 


29.9 


Bachelor’s 


19.3 


12.9 


17.5 


15.1 


22.5 


16.3 


Advanced 


2.9 


5.1 


4.2 


6.5 


9.0 


8.1 



Table 3 shows these relationships in a multiple regression framework. This regression can't be interpreted as causal, as 
it includes several factors that are probably endogenous to the home-schooling decision (e.g., parental work status and 
household income). What can be seen, however, is the relative magnitude of different influences when taken together. 
Automatic model selection routines were used to develop a pared down regression equation because some coefficients 
were sensitive to the inclusion or exclusion of other variables in the model. The initial set of variables included all 
those in Table 2, along with interactions of all variables with survey year. Two of the effects (the main effect of being 
Black, and the effect of father's education) were retained even though they didn't meet the cutoff criterion in the 
selection routine, because of their possible substantive importance. 



Table 3 

Logistic Regression of Home-school Status 
on Background and Family Characteristics: 
Pooled Data from CPS & NHES 




Regression 

Coefficient 


Standard 

Error 


t- 

statistic 


Two-parent family 


0.313 




(0.177) 


1.8 


Non-working parent 


1.337 


* 


(0.131) 


10.2 


Income squared 


-0.018 


* 


(0.004) 


-4.1 


Mother postsecondary educ. 


0.601 


* 


(0.143) 


4.2 


Father postsecondary educ. 


0.293 




(0.173) 


1.7 


Age 1 4 to 17 


0.283 


* 


(0.132) 


2.1 


Number of children in household 


0.300 


* 


(0.039) 


7.8 


Male 


-0.213 




(0.124) 


-1.7 


Hispanic 


-1.015 


* 


(0.245) 


-4.1 


Black 


-0.521 




(0.348) 


-1.5 


Black* 1994 


-1.584 


* 


(0.766) 


-2.1 


Black* 1996 


-1.750 


* 


(0.788) 


-2.2 


West 


0.461 


* 


(0.160) 


2.9 


South 


0.484 


* 


(0.146) 


3.3 


1994 


-0.472 




(0.169) 


-2.8 


Intercept 


-6.170 




(0.249) 


-24.8 


Observations 


55,204 




Null likelihood • i 


2,936.7 
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Residual likelihood 


2,606.7 




Difference 


330.1 




Model degrees of freedom 


15 





♦ Significant at the .05 level. 



Most of the same variables that showed differences across home-school status in cross tabulations were also 
significant in the regression analysis. Sex and age were retained as marginally significant. It seems that girls are 
slightly more likely to be home schooled than boys, and teenagers more likely than younger children. Household 
variables had stronger effects — family structure, mother's education, father's education, region of residence. The 
number of children in the household had a very strong effect. The main effect of income was not significant. 
However, the square of income had a relatively strong effect. This indicates that the families most likely to home- 
school their children are of middle income — neither rich nor poor. Race and ethnicity clearly had strong effects. 
Hispanics were less likely to be home schooled and Blacks were much less likely to be home schooled — especially in 
the two earlier years under study, 1994 and 1996. It seems that convergence between Blacks and Whites has taken 
place from 1994 to 1999, but the effect is not quite significant. Wc will have to await new rounds of surveys in order 
to see if this is a sustained trend. 



One of the strongest influences on home schooling from Table 3 is that of having a non-working adult in the 
household. The coefficient of there being a non-working adult is large and highly significant. The cross-tabular results 
of Table 2 gave a hint that this relationship was diminishing across years, but the interaction with year was not 
significant in the multiple regression framework. However, the main effect of non-working remains. Sixty percent of 
home schooled children have a non-working adult in the home, compared with thirty percent of other children. If 
home schooling is limited to a particular subgroup, it is probably this one. 



A major issue arising from the association of home schooling with the presence of a non-working adult is the possible 
limitations this presents to future growth. Although 40 percent of home-schoolers lived with working adults, at least 
one adult was in the labor force only part time in most cases (figures not shown). Fewer than 10 percent lived with 
two full-time working adults. If home schooling is primarily an activity undertaken by two-parent families with a non- 
working parent, it could be a self-limiting phenomenon. However, even if home schooling does remain mainly within 
this group, it has not come close to exhausting its constituency. Seven and one-half million two-adult households have 
a non-working adult at home, and the number has remained stable in recent years, despite declines in previous 
decades. More broadly, of 36 million women with children under 18, ten million do not work, and another 6.5 million 
work part time (U.S. Bureau of Labor Statistics 2000). The number of home schooled children could grow from 
790,000 to over 30 million without exhausting this core constituency. 



Is it possible that home schooling may spread beyond this core group of two-parent families with a parent at home? 
Must it also be limited to households where parents have moderate to high education? While it would seem that 
having a (well educated) parent at home would be a prerequisite for engaging in home schooling, this is not an 
absolute requirement. Many home school households have working adults and adults with low education. In all three 
surveys a small number of home-schooled children lived with a single parent or with two adults in the labor force full 
time. In addition, a small number had no adult in the home with a high school diploma. A follow-up question in the 
1999 NHES on participation in regular school by home schoolers showed that many of the home-schooled children 
who lived with working adults were also attending school at least part of the time. Still, a portion of parents remained 
who seemed to be defying logic by schooling their children at home without being home themselves. Further 
exploration of these cases might turn up special circumstances (home businesses, odd working hours, cooperative 
instructional arrangements) that could provide an explanation. Alternatively, these families could be making use of 
Internet courseware or other technologies to avoid the need for direct instruction. Many advice books and curricula 
promise home education can be successful even when parents have little time or training for the job. (Note 5) 

Geographic distribution 

One final way in which home school children differ from their peers is geographic location, as shown in Table 4. 
Home schoolers are more likely to be located geographically in places that have been destinations for internal 
migration. Using a division of the country according to migration patterns developed by Frey (2000), home schoolers 
are seen to be located in rural and suburban areas of the West which have been the recipient of migration streams 
from California and other immigration gateway states. Many of these areas have experienced explosive population 
growth. Growth, however, is not the main feature of areas where home-schoolers are found. The correlation of growth 
rate and home schooling rate of geographic areas is positive but small (around 0.2). Looking at a scatter plot of the 
two (not shown) makes it evident that home schooling is not found in booming growth areas nor in areas of decline 
but in places with moderate to high rates of growth. Nonetheless, if a person wanted to make a case that home 
schooling is on a path towards further growth, it would not hurt to point out that it is prevalent in growing areas that 
are at the leading edge of one of the major changes in migration patterns of the last few decades. Home schooling is 
tied to a broad social trend that has not yet played itself out. 
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Table 4 

Estimated Percentage of Children Home Schooled 
by Geographic Location: CPS 1994 



Geographic 

Region 


Metropolitan 

Status 


Lower 

bound 


Point 

estimate 


Upper 

bound 


White gainers 


Non -metro 


1.69 


2.34 


3.00 


White gainers 


Suburb 


1.27 


1.81 


2.34 


Melting pots 


Non -metro 


1.14 


1.60 


2.06 


Black& White 


City 


0.44 


1.00 


1.56 


Black& White 


Suburb 


0.68 


0.98 


1.28 


Slow growth 


Non -metro 


0.60 


0.80 


0.99 


Slow growth 


Suburb 


0.52 


0.66 


0.81 


Melting pots 


Suburb 


0.48 


0.62 


0.76 


White gainers 


City 


0.13 


0.58 


1.02 


Slow growth 


City 


0.32 


0.50 


0.68 


Black& White 


Nonmetro 


0.19 


0.38 


0.57 


Melting pots 


City 


0.22 


0.35 


0.49 



Geographic Definitions 



Immigrant melting pots 

California, Hawaii, New Mexico, Texas, Florida, New Jersey, New York 
Mostly White gainers 

Alaska, Idaho, Montana, Oregon, Washington, Arizona, Colorado, Nevada, Utah, Wyoming 

White and Black gainers 

Alabama, Arkansas, Mississippi, Georgia, Tennessee, Delaware, N Carolina, S Carolina, Virginia 

Slow growth/decliners 

Louisiana, Connecticut, Rhode Island, Maine, Massachusetts, New Hampshire, Vermont, D.C., Kentucky, Maryland, W Virginia, 
Pennsylvania, Michigan, Ohio, Illinois, Indiana, Wisconsin, Kansas, Missouri, Nebraska, Oklahoma, Iowa, Minnesota, N Dakota, 

S Dakota 



Attitudes toward home schooling 



The 1996 and 1999 NHES asked parents their reasons for undertaking home schooling, with 16 possible responses. 
Several themes emerge from these responses. See Table 5. First is the issue of educational quality. The parents of one- 
half the home schoolers in these surveys were motivated by the idea that home education is better education. A large 
share also viewed the issue in terms of shortcomings of regular schools: the parents of 30 percent of home-schoolers 
felt the regular school had a poor learning environment, 14 percent objected to what the school teaches, and another 
1 1 percent felt their children weren’t being challenged at school. Another theme had to do with religion and morality. 
Religion was cited by 33 percent of parents and morality by 9 percent. Practical considerations (transportation to 
school, the cost of private school) seemed of relatively minor importance. If attitudinal responses are to be believed, 
home schooling is not primarily a religious phenomenon, although religion is important. Families participating in 
home schooling do not cite cost as a barrier, even though one might presume that private schools could respond to 
their academic and moral concerns. 



Table 5 




Reasons Given by Parents for Choosing Home Schooling: 
1996 and 1999 Home Schooled Children: NHES Surveys 


Reason 


Percent 


Can give child better education at home 


50.8 







Religious reasons 


33.0 


Poor learning environment at school 


29.8 


Other reasons 


23.0 


Object to what school teaches 


14.4 


School does not challenge child 


11.5 


Family reasons 


11.0 


Child has special needs/disability 


9.0 


To develop character/morality 


8.5 


Other problem with available public/private schools 


6.2 


Student behavioral problems 


5.3 


Want private school but cannot afford it 


3.4 


Child has temporary illness 


2.9 


Parent's career 


2.2 


Transportation/distance/convenience 


1.9 


Could not get into a desired school 


1.3 



Many discussions of home school as a phenomenon refer to two classes of home schoolers — those from families with 
religious motivations and those with primarily academic concerns (Dobson 2000, Lines 2000a). To test this 
proposition, a latent class analysis was performed on the set of attitudinal questions listed above. The two-class 
model, however, provided only marginally better fit to the data than the null model. The BIC criterion, traditionally 
used to evaluate the fit of such models (see Raftery 1997), favors the null (one class) model over the two-class model. 
On the other hand, if weight is given to prior observations of two groups with two different sets of motivations, the 
two-class model might be preferred. Table 6 shows some of the characteristics of the two classes that emerge (using 
modal category extraction) from such a model. The first class of home schoolers contains 90 percent of the total, and 
resembles the smaller second class in all but a few attitudinal areas. Areas where there was a substantial difference 
between classes are shown in the bottom four rows of Table 6 (ranked from the largest to the smallest difference in 
odds of holding the attitude). The second, smaller class was more likely to name academic and other shortcomings of 
available schools, especially objections to what the school teaches, lack of challenge for the home-schooled child and 
poor learning environment. Religion was also likely to be named by the second, smaller class, although the effect was 
smaller than with the academic attitudes. 



Table 6 






Latent Class Analysis Results: 

Characteristics of Two Classes of Parents with Different Patterns 
of Reasons Given for Choosing Home Schooling: NHES Surveys 




Class 1 


Class 2 


Total percentage in class 


90.3 


9.7 


Object to what school teaches 


9.1 


60.2 


School does not challenge child 


8.9 


36.3 


Poor learning environment at school 


25.3 


64.8 


Religious reasons 


30.9 


59.8 



In summary, if there are two classes of home schoolers, they differ mostly in terms of the degree to which they 
express negative attitudes towards the schools available to them now. No simple division exists between religiously 
motivated and academically motivated parents. Due to the small sample of home schoolers available in the two NHES 
surveys, however, the evidence is still fragmentary on this point. 

Discussion and Conclusion 

Discussion 

Although the evidence on characteristics of home schoolers is still incomplete, it is important that we take account of 
these characteristics now, rather than waiting for further data collections to provide additional detail. Home schooling, 
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despite being smaller and slower-growing than claimed by some advocates, is still an important emerging 
phenomenon. What it portends for our current system of schools is still unknown. 



Home schooling has emerged with, and indeed is linked to, other emerging educational trends — on-line education and 
other systems that allow families and individuals to choose their own educational paths (school vouchers, charter 
schools). At the same time, it flies in the face of trends towards educational standardization, such as national curricula 
and systems of assessment. Another type of standardization is resulting from establishment of increasingly detailed 
systems of occupational credentialing and licensure (Adelman 2000). These trends might not be easily reconciled. 
High stakes testing, especially, has come under strong attack from home-schooling groups (see, for example. Home 
School Legal Defense Association 2001). 

The period of institutional flux now reigning in education may be the start of a departure from the 20th century model 
of regimented instruction for students entering an industrializing world. Schools seem to have lost some of their 
legitimacy as they have lost a clear functional role in preparing youth for their role in the larger economic system (cf. 
Bowles and Gintis 1976, Dreeben 1968). Rather than representing a definite trend towards "individualizing" 
instruction, however, home schooling may represent an attempt by parents to reclaim the schooling process — to make 
schooling valuable in ways that are understandable to them through the cultural means at their disposal (Swidler 
1986). This is not incompatible with Apple's (2000) description of home schooling as part of "conservative 
modernization." Yet home schooling may not be linked to a unified conservative agenda in quite the way he 
describes. There is a true tension between home educators and the school standards movement, just as there is 
between home schooling and the increasing demand by employers for occupationally specific training and credentials. 
What these movements have in common is not a conservative agenda but an attempt by each sector with an interest in 
schooling to gain greater control over the system. 



It may be that home schoolers come to create their own, new schools, as predicted by Hill (2000). It may be that home 
schoolers remain independent. In either case, however, as home schooling grows, calls will continue for existing 
public schools to provide services that cannot be provided easily by home-school families themselves — such as 
advanced courses and extracurricular activities. Lines (2000b) has shown how schools in the state of Washington have 
reacted to this challenge. They have designed special programs and learning centers where parents can often take a 
more active role in the instructional process. If this continues as a trend, schools will find themselves increasingly 
opening their doors to parental participation in ways they have not in the past. At the same time, certain families will 
be allowed to pick and choose among school offerings. The pressures on schools that might result, in an environment 
with increasing competition from other instructional providers, are easily envisioned. 

The alternative to accommodating home schoolers would involve political difficulties. First, home schoolers making 
no use of regular school facilities could not be counted on to provide political support for school funding. Second, the 
schools would lose an ally in fighting battles against standardization, test requirements and credentialing that make it 
increasingly difficult to provide a broad, general education to children. Dealing with home schoolers will require a 
difficult balance of competing claims. The success of traditional schools in dealing with the home-school 
phenomenon will depend on school leadership. 

Conclusion 

The data examined here show that it has established itself as an alternative to regular school for a small set of 
families, and is poised to continue its growth. In 1999 around 790,000 children between the ages of 6 and 17 — around 
of 1 .7 percent of the population that age — were being schooled at home, and in the late 1 990s the number was 
apparently growing. 



Home schoolers and their families were different from regular school attenders and their families, but the differences 
weren't that large. Some of the distinctive characteristics of home schoolers seemed to be decreasing. Home schoolers 
were likely to be non-Hispanic White, but there was some evidence of fading racial differences over time. Some 
distinctive characteristics of home schoolers seemed not to be changing very rapidly, but the characteristics needn’t be 
thought of as limitations to future growth. Households with home-schooled children had moderate to high education 
and income and were located in the rural or suburban West. Home-schoolers were likely to live with two adults, with 
one not in the labor force or working part time. 



We have just begun to see the emergence of home schooling as an important national phenomenon. Unless the needs 
of parents are met in different ways, it is likely that home schooling will have a large impact on the school as an 
institution in coming decades. 

Notes 

The author would like to thank Wendy Bruno for her helpful advice and Karen Kosanovich for providing tables on 
family employment trends. An earlier version of this article was presented at the annual meetings of the Population 
Association of America, Washington, D.C., March 2001 . 1 report the results of research and analysis undertaken by 
Census Bureau Staff It has undergone a more limited review than official Census Bureau publications. This report is 
released to inform interested parties of research and to encourage discussion. 
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1. A search of the ERIC database for 1999 revealed 106 citations under "charter schools," but only 47 under "home 
schooling." 

2. Due to rules of disclosure limitation, there was no complete taxonomy of metropolitan/non-metropolitan status or 
urban/rural status in the CPS files. In this research a composite measure was created, using the three way central city, 
balance of MSA and Metropolitan classification if it was available. Otherwise, MSA size was used, with over 5 
million classified as "city" and under 100,000 or non-metro classified as non-metro. 

3. Lines data were for the 1995 school year, while the CPS data were collected in 1994. 1 adjusted Lines estimates 
downward by 5 percent to represent interim growth. If growth were faster, the proper adjustment would raise the 
estimate of CPS coverage relative to state reports, making my subsequent adjustment for undercount slightly too 
large. 

4. To adjust home schooling to include non-reporting families I simply divided the CPS estimate in each state by the 
reporting rate found by Ray. Doing so provides a point estimate of well over 1 million home schoolers. However, this 
result isn't really plausible, as the bulk of the home schooled population turns up in a few states where Ray found 
extremely low rates (e.g., 0.5 million, or nearly half of all home-schoolers, in Oklahoma). I adopted a the simple 
assumption that the interview reporting rate is never lower than 20 percent. This eliminated the implausibly large 
numbers and resulted in what I believe is a fairly reasonable high-end estimate. 

5. An example of this is the recent publication of a book entitled The Complete Idiot* s Guide to Home Schooling 
(Education Week 2001). Many curriculum providers advertise their wares on the Internet and appear at home 
schoolers' conferences. 
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Abstract 

The need to develop new models for preparation of school administrators has been a prominent 
concern in educational discourse in the last decade. Having been criticized for the inadequate 
preparation of the school leadership cadre, academic departments responsible for training future 
school administrators have had to revisit their approaches and to reframe their teaching 
philosophies to ensure the readiness of their graduates for the challenges and complexities of 
school leadership. This article reports on the new model of principals' training that has been used 
in York University's Principals' Qualification Program (PQP) from the late 1990s onward. One 
component of the program brings traditional case methodology into a computer-mediated/on-line 
environment. The on-line cases are narratives from the everyday lives of the Ontario school 
administrators who serve as mentors in the on-line environment. Situating our discussion within 
the context of the rapidly changing educational landscape of Ontario, we focus on the PQP model 
to explore experientially generated case narratives as one method for teaching and learning the 
work of the local school administrator. We focus particularly on the teaching and learning 
embedded in computer-mediated or on-line case narratives used in training teachers for school 
leadership. We argue that the complexities of school leadership — the social, cultural, relational, 
ethical and moral context of school leadership — can be taught effectively through the reflective 
processes of on-line case narratives. We seek to contribute to the ongoing dialogue on the 
potential of new pedagogies and new technologies to help prepare the competent and responsible 
leaders for tomorrow's schools. 



Introduction 

The preparation of classroom teachers to be principals (local school administrators) is, at any time, a complex 
teaching process to re-frame individual horizons of professional interest and knowledge. The immediacy of classroom 
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teaching and learning that forms the lived experience of teachers must be brought into the context of relationships and 
responsibilities that extend throughout the school, its communities and the bureaucratic relations linking the school to 
the educational system. The principal must mediate these complex relationships. Indeed, complexity is one of the 
distinguishing features of the principalship. Traditionally, the work of the school principal has been to coordinate the 
relationships and tasks of the school with the community and the demands of the larger system in which the school is 
embedded. In Ontario education today, the preparation of principals for local school administration must also address 
the dramatic changes in public schooling that have been mandated by the current neo-Conservative government and 
the subsequent job unrest in the teaching profession. Thus, teaching and learning the work of the principal requires 
programs for preparing leaders that address the complexity of the position within the rapidly changing educational 
landscape of Ontario education. 



In this article, we focus on York University's Principals' Qualification Program (PQP) to explore experientially 
generated case narratives as one method for teaching and learning the work of the local school administrator. We 
focus particularly on the teaching and learning embedded in computer-mediated or on-line case narratives used in 
training teachers for school leadership. We argue that the complexities of school leadership — the social, cultural, 
relational, ethical and moral context of school leadership — can be taught effectively through the reflective processes 
of on-line case narratives. 

The Educational Context in Ontario 

In Ontario, school administrators are working in a new and difficult context. The neo-Conservative government has 
been engaged in a dramatic restructuring and marketization of the Public and Catholic education systems. Curriculum, 
assessment and budgetary authority have been centralized in the Government and the Ministry of Education. "The 
Ontario government slashed welfare benefits by 20%, higher education budgets by 15%, and public schooling by 
some 12% ... [educational funding was cut] in excess of 1 billion dollars, if higher education is included along with 
primary and secondary education" (Axelrod, 2000, p. 2, 8). Boards of Education are closing low-enrolled schools and 
user fees for school facilities have been instituted. School Boards were amalgamated to form very large administrative 
units and supervisory staff has been reduced. Curricula are being (or have been) rewritten for all levels and subjects, 
and grade-level testing has been instituted. Teacher testing is on the immediate horizon although the method and 
focus of this centralized review process is as yet unknown. Principals and vice-principals have been removed from the 
teachers’ unions, thus disrupting the collegial relationship on which school leadership has traditionally been grounded. 



Educators in the educational system in Ontario are struggling to make the pedagogical, curricular and administrative 
changes at the required pace and form. Experienced teachers and administrators were given the opportunity for early 
retirement and many did so. Ontario is now experiencing a teacher and principal shortage that is expected to continue 
for some time. The morale of Ontario educators is low and strikes and Job actions are not uncommon in Ontario 
school system. 

At the same time and particularly in the large urban centers in Ontario, the schooling context of teaching and learning 
has changed. Immigration to Ontario, particularly to the Greater Toronto Area (GTA) from other provinces in Canada 
and from around the world, continues to be at the highest rate in Canada. There are more students as well as more 
students whose first language is neither English nor French, fewer teachers, and fewer resources. Ontario's education 
was developed and has been based on White, Anglophone, and European assumptions — assumptions that no longer 
hold for huge numbers of students and their families, nor for many of the teachers within urban systems. Students, 
parents and communities of diverse of languages, race/ethnicities, abilities and family structures form the everyday 
context of many local schools. Local Board initiatives to include the teaching and learning of equity and diversity 
have been undercut by the new curriculum (which excluded words such as "equity") and the intensification of 
teachers' classroom teaching hours. 



It has been claimed that the current elected government is deliberately running the public and Catholic school systems 
into the ground so that charter schools and other private institutions will be welcomed into the education landscape 
(Barlow & Robertson, 1994). My research suggests that the traditional links between the local or regional 
organization of social class and the education system are being disrupted as Ontario re-orients its education system 
toward the requirements of a global economy (Griffith, 2000). This is the social and educational context of the current 
principals' qualifying programs. 



The Principals' Qualification Program at York University 



In Ontario, teachers who are appointed as school principals must have completed (or be in the process of completing) 
the Principals' Qualification Program (PQP). It is a required preparatory program offered by most universities in 
Ontario and, recently, by the principals' associations. The York University PQP is accredited by the Ontario College 
of Teachers and is offered through the Field Development unit of the Faculty of Education. 2 York’s PQP was 
completely revised in 1997, shifting the emphasis of the course toward relational leadership (Regan & Brookes, 

1995), leadership strategies for equity and diversity, and incorporating an action-research practicum. At any one time, 
250-300 candidates are enrolled in the program. Ten to fifteen program facilitators (current school principals) teach in 
one of the two required programs at four course sites in the Greater Toronto Area. A focus on change at the school 
and system level is integrated into the program through the seminar topics and through an on-line mentoring process. 
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Case Methodology in the York University PQP 



A small, but important feature of the York PQP (14 out of 125 hours) is taught via computer-mediated communication 
to groups of candidates who converse on-line. In contrast to the course seminars, the on-line groups are not site-based. 
PQP candidates are randomly assigned to online discussion groups (the number of the groups grew from eight in 1997 
to sixteen in 1999, to 20 in 2000). Each on-line group consists of 15 - 20 candidates who are mentored by two 
principals from different Boards of Education in the GTA. The On-Line Mentors are volunteers and represent a 
variety of Public and Catholic Boards. They are recruited for their knowledge and experience, and for their Tit’ with 
the underlying principles of the York PQP. Each week, the On-Line Mentors post cases for discussion drawn from 
their everyday experiences as principals. 



The on-line component of the program was developed for a number of reasons: 



• To provide the opportunity for ongoing learning using computer-mediated technologies, including electronic 
mail and Internet information resources. 

• To broaden the candidates knowledge of the range of issues and administrative practices of different Boards 
of Education across the GTA, for example to highlight differences between rural and urban schools, mono- 
cultural and multi-cultural schools, schools that are mono-lingual and those with a broad linguistic diversity. 



• To extend the range of contacts so useful to recently appointed administrators. 

• To provide a different learning medium for the candidates — one that is non-linear, reflective, and not tied to 
the schedules of face-to-face teaching and learning. 



Computer-mediated discussion groups allow for interactions between experienced and aspiring administrators, 
mentoring relationships that no longer need to be confined to a single classroom or school. Indeed, the on-line 
technology allows for networking across the large distances of the GTA, and provides the opportunity for people 
working in diverse school situations to learn from each other through the discussion of cases drawn from the everyday 
experience of principals working across the diversity of GTA schools. 

Case Methodology 

Case methodology has a long history in education for leadership and administration. It originated as an instructional 
technique at Harvard University Law School in 1 870 and later became widely accepted in a variety of professional 
disciplines. Clamp (web-page) defines a case as an account or description of a situation or sequence of events 
confronting an individual, a group of individuals, or an organization. Rather than deliver concepts and theories, cases 
(also called clinical correlation, leadership stories, real-life narratives, critical incidents, and vignettes) present 
situations for analysis about which decisions must be made. There are similarities of structure between case histories, 
clinical cases, problem-based learning and other experientially based methods in that they are all oriented to the 
particularities of practice while drawing on the generalities of theory. The purpose of cases and case-based instruction 
in any professional field relates directly to the nature of the body of knowledge that exists in that particular field 
(Merseth, 1997). For example, in teacher education cases are used to develop reflective practitioners and to promote 
teacher reflection and enhanced understanding (Richert, 1991), In the field of educational administration where 
decision-making skills are crucial, cases can offer opportunities to practice analysis, problem solving, action planning, 
and evaluation (Merseth, 1997). 

In the educational studies in general and in the area of school administration in particular the interest in the case 
methodology is manifested through a wide incorporation of and experimentation with case-based instruction in pre- 
service and in-service training of teachers and administrators. To date, numerous guidebooks and collections of cases 
for principalship training have been published in North America {cf: Hanson, Preparing for 
EducationalAdministration Using Case Analysis (2000); Lynn, Teaching and Learning with Cases: A Guidebook 
(1999); Miller and Kantrov, A Guide to Facilitating Cases in Education (1998), Merseth, Cases in Educational 
Administration (1997)). At the university level, special centers have been established for research and promotion of 
the case methodology in the field of education (Pace University Center for Case Studies in Education is but one 
example). Likewise, educational journals devote a great deal of attention to this matter and provide a forum for 
discussion of the value of case methodology for preparation of educational cadre. For example. The Journal of Cases 
in Educational Leadership (JCEL) published by University of Utah is devoted solely to the publication of cases that 
can be effectively used in preparation of educational leaders. 



Principal preparation programs have used case studies to remedy problems that arise in the gap between theory and 
practice. A major criticism of principal preparation programs has been: ’’Regardless of the year appointed, [principals] 
have been trained and certified as administrators through programs largely irrelevant to and grossly inadequate for the 
work responsibilities found in the school principalship” (Muse & Thomas, 1991). The use of cases that emphasize the 
"value of both theory and practice, experience and reflection” (Danzig, 1997, p. 125) were (and continue to be) 
viewed as an important way of addressing this criticism. 
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The need to bridge the worlds of theory and practice in the training of school administrators has been accentuated by 
Danzig (1997; 1999) who developed a "leadership stories" approach for university training of school administrators. 
His model is based on a mentorship through narrative where experienced school administrators offer their 
experiences — their leadership stories — to the next generation of school principals. Danzig argues that "leadership 
stories (or "narrative research") is a powerful tool for connecting the privileged discourse of universities with the 
smart hands of experience - connecting theory to practice" (quoted in Hopkins, 1998). 

According to Ashbough and Kasten (1991), cases can be used for several purposes: 

• to help students acquire analytical skills; 

• to develop the skills of synthesizing information; 

• to promote concept development; 

• to develop mature judgement and wisdom in a relatively risk-free environment; 

• to illustrate and apply widely accepted techniques of administration. 

Richert (1991) suggests that case methodology combines both artifactual and social elements. The artifactual 
component is the case itself, a description of an actual situation. The social component of case method is the 
discussion of the case by colleagues that usually takes the form of a case discussion (often referred to as a "case 
conference"). Our data suggest that case methodology also draws on the complexities embedded in the structure of 
narratives. We return to this point below. 

Using Case Methodology On-line 

The emergence of computer-based technologies opened up new possibilities for case conferencing. According to 
Desberg and Fisher (1996), the collision of technology and case methodology was inevitable. The combination of 
technology and case-based instruction gave birth to new applications of case methodology in teacher and principal 
preparation programs, for example video-cases, computer-based simulations, virtual case competitions on the World 
Wide Web, cases presented as CD-ROM and laser disks, and electronic meeting systems. Sudzina (1999) notes that 
the technological formats of case analysis can be written, spoken, or based in hypermedia utilizing the Internet or 
World Wide Web. Unlike the traditional classroom discussion of cases, electronic meeting systems (EMS) allow for 
constant ongoing dialogue between participants. Moreover, electronic brainstorming produces significantly more 
fresh ideas than traditional classroom analysis of cases (Olaniran, 1994). EMS also allows more people to interact 
without interruption and to input ideas simultaneously. Opportunity for the participation of all group members is also 
enhanced in computer-mediated case discussions. 



On-line case discussion in the York University PQP resonates with the need to decrease the gap between classroom 
instruction and practice and to enhance students' ability to make good decisions that are at the core of school 
leadership. The PQP on-line cases are experiential narratives that focus on a broad set of concerns specific to the work 
of the principal. The cases, referred to by participants as scenarios, problems or situations, are posted weekly by the 
On-Line Mentors. Each week, candidates read the posted cases and respond to the issues, events and dilemmas of the 
case taking the perspective of a principal. This requires them to shift their reading of the cases from their experiential 
teaching horizon toward a one that encompasses the relationships within the school, the school community and the 
educational system with it's diverse participants. 



The role of case discussion facilitator is important in case methodology generally and the York PQP specifically. The 
facilitator guides the discussion by probing, directly challenging, or simply observing the discussion process 
(Merseth, 1997). Feedback plays an important role in the learning process: as people receive information about the 
results of their actions, they are able to correct themselves progressively until they achieve the intended results 
(Silver, 1987). The PQP On-Line Mentors, all of whom were classroom teachers prior to becoming a principal, bring 
their knowledge of teaching and learning to the CMC environment. They provide feedback in the form of comments 
addressed to candidates' responses, suggestions for different presentation strategies, ideas for deepening the responses, 
or concluding thoughts. The On-Line Mentors also direct discussions and monitor candidates' participation in order to 
facilitate discussions that are reflective and thoughtful. In order to make case discussions challenging, educative and 
interesting, On-Line Mentors may make suggestions and ask questions related to the posted case. 



The on-line narratives are written rather than oral interactions, often presented in a laconic and point specific form 
rather than lengthy and comprehensive manner. The audiences for whom the narratives are constructed are other 
educators attending or teaching the course. Although one face-to-face meeting of the on-line groups is scheduled 
during the course, candidates may never meet their on-line mentor or other members of their on-line group. Group 
cohesion is developed through a combination of required participation, the skill of the on-line mentor, and the 
ongoing involvement of the participants. 
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Case methodology has changed in recent years from the method of instruction based on face-to-face discussion of oral 
cases/narratives confined to the classroom space to computer-mediated asynchronous discussion of on-line cases in 
virtual classrooms/cyber-classrooms. Narratives in cyberspace, rather than being a part of an oral discourse, entail 
new communicative styles and ways of engagement with texts/cases on the part of the learners. In theorizing of 
learning, claims have been made that we learn from the other's response to what we convey. Thus, learning the art of 
school leadership through on-line narratives necessitates the learners to put their responses out there and to find out 
what comes back. 

A unique feature of the York PQP cases posted on-line is that often they present an account of events that happened in 
the same week (or sometimes even the same day) as the case is posted for discussion. While varying in content and 
length, the cases are authentic narratives from the work experience of GTA principals. As such, they have some 
pedagogical similarity to more traditional principal training methods such as "inbox" and "shadowing" exercises. 

Some of the cases posted by On-Line Mentors open up with the words like: "Just a quick situation we had at our 
school this week that caused us some thought" or "Here is the situation based on a real case that happened recently." 
The fact that cases posted for discussion are "fresh" cases enables On-Line Mentors to enrich the dialogue by adding 
new details and nuances to the case and informing participants about the trajectory of events. Unlike other case study 
models that have been 'worked up' for teaching purposes, these case narratives are generated in the immediacy of the 
principals' workday and give a strong sense of the complexities of the fast-changing educational landscape in Ontario. 

Case Narratives in On-line Learning 

The literature on case study methodology neglects one of its most striking features. The case studies are narratives - 
#151;stories constructed with an audience in mind and located in a particular time and space. They are expressions of 
local knowledge, both constructing and depending for their sense on a world known in common. Rosenwald and 
Ochberg ( 1 992) claim that, "The culture speaks itself through each individual story.... Narratives are a chorus of 
coordinated stories." On-line cases, as narratives, are the chorus of coordinated stories of Ontario schools today. As 
narratives, they suggest issues that are both practical and theoretical: " Studying cases actually relies on a dialectic 
between events and meanings, practice and theory. We learn from the narrative as we reflect on the content and make 
sense of it based on what we know and believe. In studying the particular, we consider the general; similarly, we 
challenge the general by studying the particular" (Richert, 1 99 1 , p. 1 40- 141). 

Our data point to the importance of case narratives for situating on-line speakers in the stories and events of their 
profession. The use of narratives brings into view the implicit and explicit ethical and moral issues that permeate the 
stories of schooling. Narratives bring the complexity of the coordinated stories about school leadership and 
administration to the medium of on-line group learning. Equally important, they allow beginners "to consider and 
inspect the informal or tacit systems which exist side-by-side with the formal manifest systems operating in schools 
and organizations' (Danzig, 1997, p. 124). 

Teaching and Learning Complexity using Narratives 

One of the program challenges that confront PQP candidates and their program teachers are to develop the skill to see 
the case narratives through the eyes of an administrator rather than a teacher. Our prior experiences, history and 
knowledge shape our interpersonal engagement with others and form the interpretative lenses through which we 
"see" (analyze) a particular narrative. Working through case narratives with other members of the on-line group is one 
way of teaching and learning new ways of seeing the events of the school day. 

The following case narrative was posted early in the program. 

Posted case experience:You are the principal who has been trying to visit a teacher who is on the first 
year of a two-year probationary contract. Often when you scheduled a visit, the teacher was absent 
that day. Your collective agreement indicates that the principal will endeavor to give 48 hours notice 
before the teacher is observed for evaluation purposes. There have been a number of people (parents 
and students) complaining that the teacher's classroom control is unsatisfactory. Your Superintendent 
wants any terminations recommended within the week. You tell the teacher at 8:30AM that you will 
be visiting her tomorrow afternoon. She says that is "unacceptable" and walks away. What will you 
do? 

In the case responses posted early in the York PQP, candidates have difficulty seeing the complexity of cases as well 
as to sort through the diversity of approaches to a posted case — what Bridges (1992) calls the one best decision 
syndrome. Particularly during the first weeks of the course, students often headed in the direction of searching "right" 
answers rather than exploring the case complexities. This observation resonates with the findings of a growing body 
of research on the new computer mediated modalities of teaching/leaming and the challenges they pose for learners 
who are required to interact with knowledge in a new way. Research on computer mediated instruction in the training 
of school administrators suggests that students oftentimes tend to look for a right answer and thus are "likely to focus 
on surface problems rather than underlying problems. Thus, the solutions would ultimately not solve the root of the 
problem and are unsatisfactory for the school setting" (MacNeil, 1997). Addressing this issue, MacNeil (1997) argues 
that the ambiguities and complexities of school decision-making should be at the heart of computer-mediated case 




instruction and that case study methods that came to education from the fields of law and business need to be re- 
examined as a method of instruction for use in the preparation of school leaders. 

In response to the case narrative above, the candidates’ responses were diverse but focused on the administrative and 
bureaucratic Yules’ of the principalship. For example, responses were. 



• Judgmental: ’’This is a new employee, who is arrogant and does not fit the needs and profile of the school 
community.” 

• Legalistic: ’’Termination is a multi-step process beginning with a plan for improvement.” 

• Few of the responses were collegial: ’’Find out from discussion with the teacher what his/her perceptions are 
about the students. As a principal, I might already have some prior knowledge/history about the key players 
or instigators.” Even fewer focused on the school as a social and cultural unit with interdependent 
relationships across faculty, students and staff. 

• Supervisory: ”... this can be confirmed through the appraisal process. This can be targeted as an area of 
growth for this teacher.” 

In part, the tendency to assume there was one best answer to a posted case was supported by the on-line medium. A 
first response to the computer-mediated case discussion sets the frame for analysis, apparently limiting the scope of 
potential responses for other participants. Students had access to the responses of other group members that had been 
posted earlier. Often, the subsequent responses did not take up different perspectives and were greatly influenced by 
the ideas and thoughts from prior responses — a kind of group think such as that which occurs in face-to-face groups. 
One of the students summed up her frustration in the following way: ”I have read all the answers and now don't have 
anything to add.” 



As candidates continued through the program, they found ways to work towards an understanding of the complexities 
of collaboration and school-based issues. For example, one student opted for an approach that would allow her to 
respond to the cases without experiencing the influence of the thinking of her group members. "I have not read any 
answers provided thus far in fear of not being able to provide my own opinions or having them all said already.” Yet 
another candidate noted, ’’This year I am not reading anyone else’s [responses] until I have done mine.” After posting 
her responses, she read the responses of her classmates and wrote her comments that supported or criticized other’s 
responses. Regardless of the strategies candidates use to find their 'voices’ on-line, the collaborative capacity of the 
on-line medium supports candidates’ learning of the range of perspectives on a given issue as well as bringing into 
view the complexities of the issues facing school leaders. 

Ongoing feedback from the On-Line Mentors as well as subtle peer assessment are salient components of the PQP. 
They contribute to the development of the candidates' ability to address the issues in a more complex ways, to identify 
approaches that are more effective than others, and to set forward their reasons for choosing a particular approach. 
Take, for example, the response of the on-line mentor when closing the case noted above: 



Let me offer some thoughts on the situation. As principal, I realize that teacher evaluation is stressful, 
and I’m dealing with a fellow human being. I must never lose sight of that fact. She and I have 
different roles but we are two equal people. Remembering that lets me approach problem solving 
with a win-win goal in mind. 

Another on-line mentor addressed the focus on the 'right’ answer in his follow-up message to candidates: ”I preface 
this feedback with the disclaimer that I, like you, am giving my opinion. There are no right or wrong answers to this 
issue. .. I think growth lies in the debate, and the opportunity to see many different approaches to the same set of 
facts.” In the on-line mentor responses, we see a complex teaching and learning about the social and professional 
mores of the principalship. 



As the course progresses, candidates felt freer to speak from their professional experiences in the teaching profession 
and from their life experiences as people interested in education. Bringing their life experiences to bear on an 
administrative framework, students came up with diverse (often even polar) responses and felt free to disagree with 
one another. The consensual uniformity of candidates’ standpoints was less present in the analysis of the cases that 
dealt with ethical concerns or involved moral judgements. The following case is illustrative: 

The Case:k parent has phoned the Superintendent to complain that her son has been given a mark 
that is unfair. The Superintendent phoned the principal saying that the parent seemed to be making a 
good point. The principal is asked to deal with it! 



The Details:]diCk is an ”A” student who has scholarship offer from a major university if he can 
maintain his average in this, his graduating semester. The head of the geography department 
discovered that Jack had sold an essay to another student in the same class who submitted it to the 
teacher as his own. Jack has written the essay last year when he was taking a different geography 
course. Jack submitted a different, original paper that reflected his usual high standards, and was 
given a 98. When the department head realized what had transpired, both students were given a zero 
for that portion of the course (40%) which was evaluated on the basis of the essay. The parent had 
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called the department head to explain that she is a single parent who cannot afford to send Jack to 
university without the scholarship, which will be denied if Jack gets a zero. The department head 
would not change the mark and the mother appealed to the principal. The principal was still 
investigating the situation when the Superintendent called. The mother had explained to the 
Superintendent that the principal was refusing to deal with it. 

Two of the student responses to this case were: 

Jack needs to understand the severity of his behavior and the consequence that was put in place was 
appropriate. That being said, I don't believe that his future should be destroyed as a result. We all 
make mistakes and need to assume the responsibility of our behavior. Consequences are put into 
place in order to facilitate learning. Hopefully, we learn from our mistakes and move on. Some 
creative problem solving is needed here. Perhaps, Jack could be given the opportunity to make up 
this mark. Assign him another paper and allow the grade for that paper to replace the zero. 



And: 



Based on the four responses that I have read, everyone seems to be concerned with a variety of 
factors regarding Jack's zero for an essay in Geography class. My question to all of you is, however, 
quite simple: Did Jack actually commit a crime? After all, the essay he submitted was an original 
piece of work in which he received a 98. This is obviously a bright student! On this basis alone. Jack 
should receive full credit for whatever his essay was worth.. . I do not believe it to be a crime, unless 
there is a specific school policy on the selling term papers, and this is where the key lies! If no such 
policy exists. Jack didn't do anything wrong, his friend did. After all, there are hundreds of Internet 
sites where one can purchase term papers for a nominal fee. If Jack had legally set up his own 
business on the Internet, and his friend had purchased the paper electronically, is Jack still 
responsible? No way! Jack is innocent! 



The experiential ly based case narratives allow participants to connect the case to other narratives from their own 
experiences or the experiences of someone they know. This kind of reflection was encouraged by the On-Line 
Mentors who often peppered their comments, feedback, and summaries with instances from previous events evoked 
by the narrative cases and responses. For example, with reference to the case narrative of the teacher who would not 
agree to have the principal come to her classroom (above), the on-line mentor stated in parentheses: 



One of the common mistakes that I have seen principals make occurs at this stage. Instead of 
checking for data, the principal reacts to the teacher's stance by creating a confrontation and setting 
up a win / lose situation. The principal must manage to be part of the solution, not the problem. 



He then went back to the case under discussion, focusing on his plan of administrative action. 



Juxtaposing similar narratives enabled students to identify common themes and to unravel and refine the problems 
that are at the heart of the narratives from the life of educational administrators. Equally important, this process 
helped students to anticipate the consequences of their proposed actions and to generate more effective solutions. For 
example, one of the posted case narratives dealt with school closure. 



Case: You are the principal of a school that has been identified for possible closure. The demographics in your board 
suggest that there are about 6,000 excess student places in schools in your board. The government has set down a tight 
timeline which dictates that School Boards that want to increase their funding for maintenance and new school 
construction in growth areas, must identify schools by December that will close in June 1999. Although the final 
decision has not been made, the fact that your school has been targeted is of great concern. As a principal of this 
school, what role should you play in the next few critical weeks? 



Some of the participants juxtaposed similar narratives dealing with the same situation. One of the students wrote: 



Wlien I was talking to a retired principal about this case he told me that many years ago the board 
that I work for was going to convert an old elementary school in a prominent section of the town into 
the board office. Apparently the principal decided to inform many vocal parents about this in an 
effort to stop the closure. The parents were successful in keeping the school open but the principal 
found himself as Vice Principal in a smaller school out in the country. 



An encounter with a case invites the reader to forge connections between his narrative and other narratives (Shulman, 
1996). Inquiring into the relationships between cases and theory, Shulman states that to assert that a narrative is a case 
is to engage in an act of theory. To become a case, a narrative "must be seen as an exemplar of a class, an instance of 
a larger category." (p. 208). He further argues that "it appears to be a characteristic of our species that stories 
explicitly breed yet other stories and, implicitly, the categories of analysis that connect stories to one another 
conceptually. Even in the concrete act of narrative, underlying theoretical categories emerge and often become 
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explicit.” (p. 209). Therefore, relating cases to one another and relating cases to larger categories of which they are 
instances helps to answer the question "what is this case of?" and connect it to organizing principles or theories 
(Shulman, 1996). 

The complexity of narratives, and case narratives are no exception, supports the teaching and learning of the 
coordinated stories of schooling and the principal ship. A narrative always tells more than the story line, often more 
than the story teller is aware of saying (Mishler, 1 984). Working with cases, or narratives, in a group situation 
requires learning to manage the complexities of multiple perspectives and diverse points of view as group members 
work through the artifactual, social (Richert, 1991), and cultural features of the narrative. In educational 
administration where the thinking that lies behind effective leadership is complex and varied, this understanding of 
the relational, cultural and systemic levels becomes extremely important. The PQP's interactive computer-based 
discussion of case narratives emerges as "one means of socializing neophyte administrators, that is aspiring 
administrators begin thinking like the practitioners they wish to become [emphasis added] " (Ashbaugh & Kasten, 
1991). 



Traditionally, the emphasis in principal preparation programs was placed (and oftentimes continues to be placed) on 
teaching practical skills and job specific knowledge that would allow neophyte administrators to cope successfully 
with challenges and demands of principalship. According to Mitchell and Tucker (1992, p.30), "educators tend to 
think of leadership as a matter of taking action and getting results." Less prominent has been a concern for an ethical 
dimension of school leadership which in part can be explained by the fact that in literature on school administration 
the work of school leaders was compared to and consequently viewed through the lenses of effective management and 
problem-solving skills rather than through the perplexities and ambiguities arising from the need to make decisions 
and choices that have significant ethical ramifications for all of those involved. In the last decade, the evolving 
concepts of leadership attempted to address the previously overlooked and under-theorized aspect of ethics of school 
leadership. Among the theories that explored the "ethics-leadership" nexus were the concept of moral leadership, the 
ethics of caring, transformational leadership, to name a few. These theories testified, at least to some extent, that 
"leadership is less a matter of aggressive action than a way of thinking and feeling — about ourselves, about our jobs, 
and about the nature of the educational process" (Mitchell and Tucker, 1 992, p. 30). 

Despite the recognition (at the theoretical level) of the importance of addressing the issue of ethics in the preparation 
of school leaders, this question has continued to receive only peripheral attention at the level of practice and, by and 
large, was not incorporated into the formats of principal preparation programs offered by universities and principals' 
associations. It is within this context that narratives in general, and on-line narratives in particular, emerged as a 
pedagogical space for exploration of the issues of ethics as it relates to school leadership. As such, on-line narratives 
provide an opportunity to shift the focus from teaching skills to teaching reflection and thinking. Narratives that 
present explicit (or otherwise) ethical dilemmas and challenges faced by school leaders require a clear vision and 
ethical commitment on the part of those in the position of decision-making. Narratives become a powerful 
pedagogical tool for helping future administrators to "think and feel" (rather than to judge and fix), to reflect and 
listen, to challenge widely held assumptions and biases and finally, to shape and articulate the core values. In his 
article on leadership stories, Danzig (quoted in Hopkins, 1998) notes, that "Professionals need to understand not only 
the technical aspects of their job but moral basis of their work. Stories provide a more complete view of the meaning 
of professional practice." Indeed, calling for articulation of individual moral stances, narratives offer a pedagogical 
space for engaging with the aspects of ethics as it relates to the everyday work of school administrator, and, more 
broadly, to the mission and goals of school leader at this historical juncture. 



Returning to our analysis of the PQP on-line narratives, student responses to less complex and more transparent cases 
often resulted in producing collective virtual texts where each ensuing individual script, more or less, echoed the line 
of thinking developed in responses posted earlier. In contrast, the responses to case narratives that drew on local 
experience with ethical issues embedded in the stories did not turn into unitary, virtual texts — one-best solution — 
produced by collective efforts of on-line participants. Instead, those cases generated the breakdowns and 
discontinuities of the texts and the plurality of individual standpoints. Rather than searching for a common ground and 
attempting to fit their responses into the Procrustean bed of emerging collective narrative, the students were 
challenged to 



• voice their positions, 

• develop their on-line identities as future school leaders, 

• produce their idiosyncratic texts that did not necessarily fit into the evolving trajectory of thinking about the 
posted case, 

• and finally, produce polyphonic virtual texts in which the center of gravity was shifted towards diversity of 
opinions and plurality of voices. 



Concluding Comments 



Our analysis of the on-line narrative cases and responses of the on-line groups in York University's PQP shows some 
of the features that contribute to an effective use of case studies in an on-line environment. While York University's 
PQP has a variety of pedagogical forms including course seminars, an action practicum and the on-line discussion 
groups, a cornerstone of the program is reflective practice. The Program Facilitators who have the major 
responsibility for the program have built reflective practice into each section of the course. The case narratives and the 
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interaction of the On-Line Mentors support and enhance the reflective features of the PQP within their group. The 
case narratives are drawn from the immediate school experiences of the principals who serve as On-Line Mentors and 
the character of such case narratives engages the student in a reflective process. Posted cases situated in the life of the 
school construct the case narrative as an event worthy of reflection. The necessarily-reflective character of the on-line 
discussion groups build on the social and cultural complexities of the narrative format in interesting, non-linear ways. 



The indicators of successful utilization of computer-mediated communication in the PQP are two-fold. First, there is a 
high level of candidate participation in the discussion of cases. While 14 hours of on-line participation is a course 
requirement, we have found that many of the candidates participate in the on-line discussions for more than the 
required hours. Second, despite their workload and variety of commitments, few On-Line Mentors withdraw from the 
program. The PQP archives show that the mentor / principals were positive about their role as On-Line Mentors as 
opportunities for learning and professional growth. The following response exemplifies the attitude of the principals 
participating as On-Line Mentors in the PQP: "This is my third year mentoring Principal candidates at York 
University. The reason that I do this is to increase my own learning and test the effectiveness of electronic 
communication." (A Principal in a grade 7-OAC school). 



Our analysis points to two other features of case study methodology. First, cases that link to the professional 
experience of teachers provide the cognitive basis for generalizing from local experience to the more-general 
perspective required of school principals. In contrast to the uncertainty and subsequent ’group-think* that occurs when 
candidates must respond from within the conceptual framework of educational administration, when candidates are 
able to construct responses based on their experience the resulting diversity of response and debate provides fertile 
ground for teaching by the on-line mentor. Second, encouraging narrative forms would seem to enhance the 
complexity of the on-line responses. Beginning in the everyday experience of school administrators and constructing 
case studies that evoke narratives that include ethical and moral issues would seem to be an effective way to teach 
leadership. 



In sum, the texts produced via the medium of technology become excellent "raw" material for thinking about and 
reflecting upon the nature of the school leadership. They push the students to explore the limits of their knowledge, to 
test the strengths and merits of their beliefs, to encounter other views, to share values and to reflect upon the 
vicissitudes of the school principalship. Pedagogically speaking, narratives that include moral and ethical issues are an 
effective way of teaching the art of school leadership and of developing and strengthening the sense of ethical 
responsibility in future educational leaders. 

Notes 

^Article prepared for the Fifth Annual Leadership Conference on Ethics and Values, Bridgetown, Barbados, Sept 
25 — Oct 1, 2000. We would like to thank Terry Gray, the volunteer On-Line Mentors and the Program Facilitators of 
our principals’ program for their excellent teaching and their comments on this work. 



^The York PQP does not carry university credit. 



^These similarities were brought to our attention by one of the principals teaching in the York PQP who had 
previously been an On-Line Mentor. 



^As with all narratives, the on-line cases are located in time and space — within a particular course context that shapes 
the boundaries of the stories. The course philosophy attends to those features of educational administration that deal 
with human experiences and interactions, suggesting that the relational character of leadership is as important for 
administration as are specific educational acts and regulations. The majority of electronic archived cases from the 
program year that we selected for our analysis are narratives that emphasize the complexity of people’s behavior and 
interactions. The cases and responses emphasize the skills of problem solving, analysis and critical and creative 
thinking rather than specific knowledge of any particular educational issue. 



^The data for this article were drawn from (archived) on-line discussions between PQP candidates and volunteer 
mentors during one of the recent program years. 
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This article addresses the question of how much is spent on teachers' professional development. 
A review of the literature finds two problems that have frequently led to inaccurate estimates of 
professional development spending: 1) the accounting codes that are used in many studies 
provide little description of spending, and 2) studies generally focus on district or state 
expenditures for professional development, but do not collect data on school-level spending. 
These problems are compounded by the fact that studies define professional development 
spending differently, and thus it is difficult to compare findings across studies. In an effort to 
begin to address this problem, this study utilizes a detailed cost structure to analyze both district 
and school site expenditures on professional development across cost categories. The study 
found that school-level expenditures were a significant source of professional development for 
teachers. This has implications for the methodologies used to estimate current professional 
development expenditures and what level of expenditures would be necessary to generate 
dramatic improvements in student achievement. 



In a climate of standards-based reform, schools are being called upon to assure that all students achieve to high 
standards. While most experts agree that extensive staff development will be necessary to improve instruction so this 
goal can be realized (Birman et al., 2000; Corcoran, 1995; Hertert, 1997; Killeen, Monk and Plecki, 2000; Little, 
1993), few studies have identified effective professional development and even fewer have documented the 
professional development costs associated with implementing powerful, focused reform. This article shows how 
building knowledge about the level and structure of professional development spending necessary to achieve the goals 
of standards-based reform requires three changes in research design. First, a methodology needs to be developed that 
improves on traditional data used to identify spending. Second, it is necessary to have an analytical framework to 
understand the types of spending on professional development. Finally, it is important to accurately estimate the 
professional development resources available at schools, which are frequently the focus of standards-based reform and 
which can make decisions to augment or decrease the professional development resources available to teachers. 
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Abstract 



This case study extends a methodology developed by Hawley-Miles (Miles et al., 1999; Miles and Hombeck, 2000) 
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and her colleagues to collect district-level data on professional development spending to the school level and analyzes 
that data using the professional development cost structure developed by Odden, Archibald, Fermanich and Gallagher 
(2001). This case study was part of a larger research project that examined several urban districts’ spending on 
instructional improvement. It is an early step of a broader research program that seeks to determine the level of 
professional development spending and spending strategies that will lead to improved teaching and higher student 
achievement. This case highlights Elm Street School, a K-8 school in a large urban district in the Mid-West that used 
professional development as an integral part of a coordinated schoolwide reform. Professional development is defined 
here as any activity intended to build teacher knowledge, skills and classroom instructional expertise. This includes, 
for example, workshops, teacher coaching, work with consultants, and the cost of teacher time to participate in 
activities designed to lead to professional growth; since this case study examines district and school expenditures, it 
does not cover activities that have no cost for the district or school (for example, course-taking paid for by teachers). 



This case study seeks to answer three questions: 1) What is Elm Street’s strategy for improving student achievement? 
2) How does professional development support this strategy? 3) How does Elm Street allocate resources for 
professional development? In an attempt to answer these questions, prior research on professional development costs 
was reviewed. The first section provides an overview of the findings, along with an analysis of the strengths and 
weaknesses of those studies. The analysis leads to an explanation of the methodology used to understand Elm Street’s 
resource allocation. Next, Elm Street’s educational strategy is explained in light of its district context. Finally, Elm 
Street's professional development spending is examined in terms of funding sources and the Odden et al. (2001) cost 
structure. 

1. Prior Research 

This section discusses eight studies on professional development spending. Taken as a whole, these studies highlight 
three important issues in this area of research: 



• One common source of data is fiscal accounting codes. Studies that rely on this data source generally do not 
have detailed descriptive information about the nature of the professional development that was purchased. 
Other studies have sought to avoid this problem by using alternate data sources and focusing on the type of 
professional development provided to teachers. These studies have generally lacked detailed information 
about professional development spending. 

• Researchers have used different analytical categories when reporting on professional development spending. 
As a result, it is difficult to make accurate comparisons across studies. 

• Most studies have focused on spending at the district level. Few, if any, studies provide information about the 
professional development resources available at the school level. Since schools can be the locus of much 
decision-making about professional development, the lack of knowledge about school-level professional 
development spending may lead to inaccurate estimates of professional development resources available to 
teachers. 



The studies reviewed in this section place the current case study in the research context by highlighting methodological 
issues and also showing what is currently known about professional development spending. 



In one of the first major studies on the costs of professional development. Little et al. (1987) used interviews, surveys 
and state documents to analyze professional development spending in California in terms of school, district, regional, 
and state expenditures for participants' time and for the cost of providing the professional development activity. This is 
a landmark study because of its focused analysis on the quality of professional development and because it attempted a 
comprehensive analysis of professional development spending. The authors found that, on average, professional 
development spending equaled approximately 5% of the total classroom costs (or $4,379 per staff member; $6,880 in 
2000 dollars). 

The analysis provides a good large-scale picture of professional development spending. However, the estimates 
include the cost of two items that make them problematic: uncompensated teacher time (worth an estimated 60 cents 
for every dollar spent by the district on professional development); and lane salary increases resulting from credits 
earned through professional development activities (estimated as 61% of total staff development costs). These two 
items dramatically increase the estimated expenditures, yet uncompensated teacher time is not a direct cost. More 
importantly, salary increases are a legal part of teacher contracts and could not be reallocated to professional 
development strategies. Classifying these expenditures as professional development expenses is not useful for an 
analysis like the present one, which is focused on the actual dollars available for professional development at the 
school site. With the present value of semester credits and uncompensated teacher time excluded from the analysis. 
Little et al. found that professional development accounted for around 1.4% of total classroom expenditures (or $1,229 
per staff member; $1,93 1 in 2000 dollars). It is also important to note that although Little gathered data on "investment 
in school-based staff development," she found comparatively few of these expenditures. Since the report does not 
include survey instruments and interview protocols, it is unclear if such spending was not found because she did not 
look for school-level discretionary spending or if schools lacked sufficient control over their own budgets to provide 
professional development. 

Little’s use of non-budgetary documents allowed her to avoid some of the difficulties that arose in many other studies 
on professional development expenditures, which utilized budgets as the sole data source. As Chambers (1999) has 



argued, since accounting codes track resources only by source and expenditure categories created for fiscal accounting 
purposes, they do not provide meaningful information about many types of expenditures and their results. The lack of 
descriptive data in typical fiscal accounting systems created difficulties in several studies of professional development 
spending. 

One study that encountered this difficulty was Hertert's (1997) multi-district analysis of professional development 
spending. Hertert attempted to use district budget data to estimate state and district professional development 
expenditures, evaluate the connections between spending and improvements in student performances, and suggest ways 
of reallocating resources to the most effective types of professional development. She originally approached 60 
districts for participation, however only 16 kept the data necessary for a basic cost estimate. Of those, none were able 
to furnish information that would allow her to address the second and third questions in her study. Hertert was able to 
analyze professional development spending across six categories: professional development office, district 
conferences/workshops, nondistrict conferences/workshops, inservice training days, university/college coursework, 
sabbaticals, and temporary assignments. The analysis showed significant variation in districts' professional 
development spending, ranging from 1.7%-7.6% of net operating expenditures, with an average of 3.6% across 
districts. The spending level was equivalent to an average of 6.8% of the cost of teacher salaries, including benefits. 
From this she estimated that per teacher spending would be equivalent to $3385 if the district paid its average teacher 
$50,000 including benefits ($3,825 in 2000 dollars). Her study is most important, however, from a methodological 
perspective because it highlights how accounting mechanisms hamper research on professional development spending 
and the connection to educational outputs, which were the unanswerable parts of Hertert's original question. 

Killeen, Monk and Plecki (2000) also attempted to understand districts' spending on professional development but 
chose to use two major national datasets, the Census Bureau’s Survey of Local Government Finances: School District 
Finances (F-33), and NCES's Common Core of Data-Longitudinal File to produce nationally generalizable results. The 
study analyzed expenditures categorized as 'instructional staff support,' which provided the nearest approximation to 
professional development. Unfortunately, in one dataset this category also included items such as library, television, 
audio-visual, and computer-assisted instruction, which are not professional development. Further, much central office 
instructional support is also supervisory activities, which do not fit into a general definition of professional 
development. In conjunction with Hertert's work, this study clearly shows some of the challenges of attempting to 
quantify professional development spending in districts and schools using the data available in current fiscal 
accounting systems. 

Killeen, Monk and Plecki (2000) found that district spending for professional development ranged from 1.27-8.10% of 
total general district expenditures, with the average district spending 2.76% percent of its budget for professional 
development. While this range may seem large, most states’ average district spending on professional development 
was between 2-5.2%, with only six states averaging below 2% and only Kentucky averaging above 5.2%. The average 
per pupil expenditure on professional development across states was $192 ($223 in 2000 dollars). 

Other studies have discussed professional development costs but have focused more on describing and analyzing the 
nature, and extent of professional development opportunities. Caret et al. (1999) conducted a major study of 
professional development under the Eisenhower program, a federal math and science education initiative under Title II 
of the Elementary and Secondary Education Act. Eisenhower funding sought to improve students’ math and science 
achievement through teacher training. Caret et al. conducted a nationally representative survey of teachers 
participating in the Eisenhower Professional Development Program to determine the characteristics of effective 
professional development, which they defined as professional development that leads to changes in teacher knowledge 
and practice that produced increases in student achievement. They found that effective professional development has 
six main features: 

1 . form (professional development should be school-based and job embedded); 

2. duration (long-term and ongoing professional development is better); 

3. collective participation (it is beneficial to have groups of teachers from the same school or department share 
the professional development experience); 

4. content focus (teaching strategies should be combined with enhanced content knowledge of what is being 
taught); 

5. active learning (opportunities for teachers to become engaged in their own learning are important); 

6. coherence (professional development should be aligned with state standards, assessments, teachers' goals and 
school and district context). 

They found that few teachers participated in highly effective professional development and that one of the main 
reasons was the higher cost. While districts spent an average of $185 per teacher ($197 in 2000 dollars) on typical 
professional development under the Eisenhower program, they found that exemplary projects in the Eisenhower 
Program spent approximately $512 per teacher ($529 in 2000 dollars) to provide effective professional development. 
The increased cost was typically for providing professional development of longer duration, and frequently included 
more active and embedded learning than the workshops that characterize traditional professional development. Though 
the study provides very useful information on the features of effective professional development, there is little 
explanation of how these costs were determined, which limits the usefulness and generalizability of the cost estimates. 
Furthermore, the estimate is for professional development in only one subject area, and must, therefore, be seen as a 
probable underestimate of the overall cost of professional development. 




In their study of professional development in New York City's District 2, Elmore and Burney (1997) found that the 
district spent approximately $1 ,300 per teacher ($1 ,427 in 2000 dollars) on professional development, or 3% of the 
district's operating budget. Elmore and Burney provide more detail than most on how the money is spent, by dividing 
overall spending into four categories: teacher compensation, contracted services, professional development lab, and 
materials. Yet they do not provide enough detail to analyze the expenditures in terms of how money was spent within 
those broad categories. For example, it is impossible to determine how much teacher compensation spending was for 
stipends for teachers attending professional development activities as opposed to the cost of substitutes to provide 
teachers release time. Additionally, one might want to know within contracted services, how much was spent on one- 
time workshops as opposed to ongoing coaching. Since other components of the ongoing study in District 2 
demonstrate positive outcomes from the district's strategy, it would be very useful to have a systematic cost 
methodology so that District 2's spending on various professional development strategies could be compared to other 
districts. 

Miller, Lord and Domey (1994) presented a cross-case analysis of professional development in four districts. They 
used district-lcvcl interviews, principal interviews, and teacher interviews to build an in-depth understanding not 
available through analyzing budget data alone. However, they only presented a rough estimate of the percentages of 
spending on professional development. For example, in analyzing salary costs, they estimated that 15% of principals' 
time was a professional development cost, yet provided no explanation for how they arrived at this estimate. Their 
results are shown in Table 1 . 



Table 1 

Miller, Lord and Dorney's Estimates of Professional Development Spending 



District 


Cost per Regular Classroom Teacher at time of 
the study (in 2000 dollars) 


Cost as a Percentage of 
Operating Budget 


Large 


$ 3,529 ($4,462) 


2.3% 


Large 


$ 1,755 ($2,219) 


1.8% 


Medium 


$ 2,706 ($3,421) 


2.0% 


Small 


$ 3,528 ($4,461) 


2.8% 



They broke down this spending into six categories: baseline (staff development office); district and school-level staff 
development salary; materials and services, travel, consultants, and miscellaneous; substitutes; externally funded 
programs; and personal contributions. These categories add little to our knowledge about how money is spent and since 
they did not explain in detail how these were derived and did not analyze the spending categories within externally 
funded programs, it is difficult to make meaningful comparisons to other studies. 



Miles (Miles et al., 1999; Miles and Hombeck, 2000) presented a more detailed approach to tracking districts' 
professional development spending. In a study of Boston's professional development spending, Miles, et al. (1999) 
analyzed professional development spending in light of two key concerns: 1) how closely professional development 
resources were aligned with the district's improvement plan; and 2) how well they matched the National Partnership for 
Excellence and Accountability in Teaching (NPEAT) principles for effective professional development. They began 
by collecting budget data from all sources, and tentatively coding this data by district role (e.g. professional 
development, accountability, curriculum development and support), function (e.g. salary, stipend), and source (federal, 
state, local, private). They then interviewed heads of all relevant district departments to determine what activities were 
related to professional development. This allowed them to include costs, such as the salary for staff who designed 
professional development, that are not included in some analyses. Their interviews also enabled them to gather data 
on: 



• The sources of professional development funds; 

• The type of professional development activities purchased (e.g. consultant, staff salary, etc.); 

• The topic of professional development activities; 

• The locus of control for the professional development funds; 

e The percentage of time district personnel spent on developing or providing professional development. 



These data gave Miles et al. a refined understanding of Boston's professional development spending that enabled them 
to analyze spending by type, topic, control and source. They determined that the district spent over $23 million per 
year ($4,894 per teacher and principal; $5,170 in 2000 dollars) on professional development, or 3.8% of their total 
budget. Furthermore, by comparing this spending to district goals and NPEAT's principles for effective professional 
development. Miles et al. were able to make recommendations for how Boston could reallocate resources to improve 
the effectiveness of their spending. 

In later work. Miles and Hombeck (2000) expanded on this methodology to compare spending on professional 
development across four urban districts and the broader concept of instructional and school support across two of those 
districts. Their definition of instructional and school support included all activities undertaken on the part of a school 
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district to support high quality instruction: professional development, accountability, curriculum development and 
support, special program monitoring and compliance, information systems, district student services and community 
outreach. These were included to the degree that they supported instructional improvement. For example, spending on 
information systems that allows schools to better analyze student performance data and tailor reform to specific needs 
would be considered instructional support; information systems spending that went to monitor student attendance 
would not be included. Miles and Hombeck used interviews with department heads and other key personnel to 
determine how much each department spent on instructional support, and combined the data to learn how much the 
districts spent on professional development and instructional support. 

They found that there was substantial variation across the districts in terms of the overall level of professional 
development spending as well as how money was spent. The districts spent between 2.4%-4.3% of their operating 
budget on professional development, not including the cost of contracted inservice training days. When these were 
included, the range was 2.4%-5.9% of the district operating budget, or from $2,010-$6,628 per teacher ($2,078-$7,002 
in 2000 dollars). Additionally, district spending was frequently fragmented across many departments rather than 
focused on the districts’ highest priority areas. Finally, district spending differed by strategy. While some districts 
invested heavily in workshops or subsidizing university course-taking, others spent a higher proportion on stipends for 
teachers to take on responsibilities outside of traditional teaching. 



While Miles’ studies provided significant detail on district costs and strategies, they did not trace district expenditures 
to the school level. In districts that have decentralized school funding, the school general fund budget, as well as any 
other budgets controlled at the school level, could be an additional source of professional development spending. By 
extending Miles’ methodology to include a school-level analysis, this case study of Elm Street provides a more 
complete picture of the resources used at a given school. 

As can be seen, past studies on the costs of professional development have struggled with at least one, and usually 
more than one, of the following issues: 



• The study provided information about professional development activities but provided little information about 
costs; 

• The study identified costs, but lacked rich data on the nature of the professional development activities, or data 
on the different categories of expenditures that comprise total costs; 

• The study identified costs, but did not use a systematic methodology that enables comparisons to other 
research; 

• The study had no data on how schools supplemented or reduced district-level professional development 
resources available to teachers. 



The first problem has created an overall scarcity of information about professional development expenditures and 
costs. The second is quite pervasive in the literature because analyses of school and district spending have typically 
tracked resources from source to accounting code expenditure, which provides little information on the nature of the 
professional development spending. The lack of a systematic methodology for identifying costs is even more 
problematic because of the first two problems; since few studies provide a desirable level of detail on both strategies 
and expenditures on professional development, it would be highly beneficial to be able to make meaningful 
comparisons across studies. 



Finally, the majority of studies have analyzed district spending on professional development, since districts have 
traditionally been thought of as the source of most staff development resources. Schools, however, are increasingly 
playing a prominent role in supporting instructional improvement. They serve as the site of many of the more 
innovative professional development strategies, like on-site coaching and peer mentoring, which are more likely to 
have the characteristics of effective professional development identified in Caret, et al. (1999). Many schools also 
have control over at least a portion of their budget and are allocating some of these resources to professional 
development. Conversely, they can also choose to disregard recommendations for spending money on professional 
development instead purchasing something else, thus reducing professional development spending. For these reasons 
it is necessary to include school-level professional development expenditures in the analysis of how professional 
development resources can be used to improve instruction and student learning. This study addresses these issues. 

2. Framework 

Based on prior research, the following goals were set for the study: to develop a methodology that would provide a 
good estimate of the total professional development expenditures at the school site and information on the nature of 
professional development activities; to use a systematic framework for analyzing professional development costs. 



This case study builds on Elmore and Burney’s (1997), Miller, Lord and Domey’s (1994), and Miles’ (Miles et al., 
1999; Miles and Hombeck, 2000; Miles, 2001) methodologies, but takes them a step further by tracking district 
spending on professional development to the school site, collecting data on school-level expenditures for professional 
development and analyzing expenditures in terms of a clearly articulated cost structure. Even though the resource 
picture at Elm Street is quite complex, the methodology outlined below made it possible to develop an in-depth 
understanding of the sources and deployment of professional development resources at Elm Street. The next section 



explains the cost structure that was used to analyze professional development resources from both levels. 

In a review of literature on the costs of professional development, Odden, et al, (2001) built on the Caret et al, (1 999) 
and Elmore and Burney (1997) studies (among others) to create a systematic framework for analyzing the costs of 
professional development. Looking across these existing studies they identified six types of school and district 
professional development expenditures: 1) teacher time, 2) training and coaching, 3) administration, 4) materials, 
equipment and facilities, 5) travel and transportation, and 6) tuition and conference fees. This cost structure provides a 
way to identify, calculate and analyze the professional development resources that districts and schools make available 
to teachers at a given school site. Table 2 presents the Odden, et al, (2001) cost structure, which is used in the 
remainder of the article to identify and analyze professional development expenditures at Elm Street, For a more 
detailed explanation of the elements of the cost structure and a general example of how to calculate expenditures see 
Odden, et al. (2001). 

As will be seen in this case study, the cost elements provide a meaningful level of detail on how money is spent for 
professional development at the district and school level. It covers all expenses necessary to produce and carry out a 
broad range of professional development activities. The usefulness of this sort of framework for making comparisons 
across studies becomes most apparent when analyzing the studies by Miller, Lord and Domey (1994), Miles, et al, 
(1999) and Miles and Hombeck (2000). Although these studies use somewhat similar methodologies, it is difficult to 
draw conclusions across studies about the level and effectiveness of professional development spending without a 
shared analytic framework. The next section explains how data were collected for this study. 



Table 2 

A Cost Structure for Professional Development 



Cost Element 


Ingredient 


How Cost is Calculated 


Teacher Time Used 




Time within the regular 


1, 


teachers' hourly salary times 


for Professional 




contract: 




the number of student free 


Development 


1, 


-when students are not 




hours used for pd 






present before or after 


2. 


the cost of the portion of the 






school or on scheduled in- 




salary of the person used to 






service days, half days or 




cover the teachers' class 






early release days 




during planning time used 
for pd 




2. 


-planning time 

Time Outside the regular 
day/year: 


3. 


the stipends or additional pay 
based on their hourly rate 
that teachers receive to 








compensate them for their 
time 






3, 


-time after school or on 
weekends 


4, 


substitute wages 




4, 


-release time provided by 
substitutes 






Training and 
Coaching 




Training 








1, 


-salaries for district trainers 


1, 


sum of trainer salaries 




2, 


-outside consultants who 
provide training; may be 


2, 


consultant fees or 






part of CSRD 




comprehensive school design 
contract fees 






Coaching 








3, 


-salaries for district coaches 
including on-site facilitators 


3, 


sum of coach and facilitator 




4, 


-outside consultants who 




salaries 






provide coaching; may be 
part of CSRD 


4, 


consultant fees or 
comprehensive school design 
contract fees 
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Administration of 

Professional 

Development 


1 . Salaries 

2. Overhead 


1 . Salary for administrators of 
professional development 
programs times the 
proportion of their time spent 
administering the programs 

2. Overhead or supplies 
necessary to administer 
programs 


Materials, Equipment 
and Facilities Used 
for Professional 
Development 


1 . Materials 

2. Equipment 

3. Facilities 


1 . materials for pd, including 
the cost of classroom 
materials required for 
CSRDs 

2. equipment needed for pd 
activities 

3. rental or other costs for 
facilities used for 
professional development 


Travel and 
Transportation for 
Professional 
Development 


1 . Travel 

2. Transportation 


1 . Costs of travel to off-site pd 
development activities 

2. Costs of transportation 
within the district for 
professional development 


Tuition and 


1 . Tuition 


1 . Tuition payments or 


Conference Fees 




reimbursement for 




2. Conference Fees 


university-based pd 






2. Fees for conferences related 






to pd 



3. Methodology 

The data collection for this study began in conjunction with Miles' (2001) multi -district analysis of expenditures on 
instructional improvement. First data were collected on instructional and school support at the district level. As in 
Miles’ earlier work, instructional and school support was defined as all district supports for high-quality instruction, 
including professional development. The analysis began with the entire district general fund budget as well as those 
from all other public and private sources of funding for the district. Line items such as transportation costs, which 
were clearly unrelated to instructional improvement, were eliminated. The remainder of the analysis had six main 
steps: 

• a) District-level interviews were used to develop an understanding of which expenditures were related to 
instructional and school support and to code spending in all departments within the various categories of 
instructional support: professional development, accountability, curriculum development and support, special 
program monitoring and compliance, information systems, district student services and community outreach. 
Interviews were conducted with the people in charge of many departmental and categorical budgets including: 
quality improvement, career in teaching, administration, curriculum & assessment, magnet, vocational 
education, accountability, teacher leadership, professional development. Title I, Title II, and special education, 
among others. The interviews provided data on which district initiatives supported instructional and school 
support, the type of spending each related line item represented, and the percentage of salary costs for relevant 
individuals that should be considered instructional and school support. 

• b) At this point, the focus narrowed to those expenditures within instructional and school support that had been 
defined as professional development. This included, for example, district literacy coaches, stipends paid by the 
district for lead teachers, the costs of comprehensive school reform design contracts, salary costs for those 
coordinating professional development, consultant fees, materials costs, and the district's professional 
development center. The analysis includes the cost of teacher time within and outside of the regular contract. 
In this case, it did not include teacher inservice days, since the district has none. As explained earlier, the cost 
of salary advancements due to professional development credits and the cost of uncompensated teacher time 
were not included. 

• c) For each line item, several types of data were collected: the description, source, control (e.g. district, 
school), type (e.g. consultant fee, stipends), topic (e.g. literacy, standards), form of delivery (e.g. school-based 
coaching, workshop). With this level of detail, it was possible to sort data according to general initiative (e.g. 
literacy, standards, teacher leadership) as well as by cost element (teacher time, training and coaching, 
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administration, materials, equipment and facilities, travel and transportation, and tuition and conference fees). 

• d) Professional development costs from all district budgets were then allocated, where possible, to the school 
level. The cost of each initiative was divided by participating schools based on the staff and overhead costs in 
one of three ways; 

o By participating school — for example, if twenty schools participated in a literacy program, the 

overhead costs for the entire coaching program would be split evenly amongst the twenty schools. If, 
within that program, five schools shared a literacy coach, each school was ’charged’ for 1/5 of that 
coach’s salary. 

o By pupil at each participating school — for example, one initiative provided a block grant to 

participating schools based on student enrollment. The number of pupils at each participating school 
was multiplied by the per pupil funding formula to determine the resource level at the school; 

o By participating teacher — for example if the district offered an after school workshop that cost $ 1 ,000 
to produce and five teachers attended, each of their schools would be ’charged’ $200. 



The precise method for doing this for each initiative is explained in a later section. 



• e) Once district-level spending was tracked to the schools, resources from the site budget and categorical 
programs were analyzed. Line item budgets were available for both the school general fund and Title I (the 
federal grant that provides supplemental educational funding for low-income students) budgets. For Title VI 
(the federal class size reduction initiative), Obey-Porter (the federal comprehensive school reform 
demonstration project). Literacy Today (a state literacy initiative) and TechNow (a state technology initiative) 
only total allocations were available. All of these sources were added to the district information to generate a 
preliminary estimate of resources controlled by the school. The next step ascertained how much of these 
resources were used for professional development. 

• f) Using the data collected in the earlier steps, preliminary and follow-up phone interviews were conducted 
with the principal to verify information, identify how categorical dollars were used and determine how the 
school allocated discretionary dollars for professional development at the school site. The interviews also 
provided an understanding of the school’s educational strategy and how resources were deployed to achieve 
school goals. At all stages, data was gathered by cost structure elements. 



These data provided three types of information: Qualitative information on the school’s goals and strategies; A 
comprehensive resource use picture; Descriptive data that enabled this analysis to move beyond accounting codes to an 
understanding of the professional development strategies and their cost. 



The combination of these three types of data makes it possible to present professional development spending at Elm 
Street School utilizing the cost structure developed by Odden, et al. (2001). Additionally, as becomes apparent in the 
next section, these allow for a transparent explanation of how cost estimates were developed. Finally, this 
methodology and cost structure makes it possible to overcome the barriers typical accounting practices create to 
understanding the relationship between professional development and educational strategies. 

4. District And School Context 

Elm Street School is in the Cincinnati Public Schools, a moderately large, urban district in the midwest. In the mid 
1990s the district had relatively low achievement, but has made a significant attempt to generate improvements by 
focusing on school accountability, coordinated reform, teacher leadership, and instructional improvement. Cincinnati’s 
accountability system categorizes schools into five performance categories, the lowest of which can trigger ’redesign,' 
the district’s school reconstitution plan. Redesign schools receive a new principal, who hires lead teachers. Together 
they select a new staff, who are required to implement the comprehensive school reform model chosen by the school’s 
redesign committee (made up of four members chosen by the district and four members chosen by the teachers’ union). 
Cincinnati has also supported the adoption of comprehensive school reform models in many schools in the district that 
are not redesign schools. 



Cincinnati has several other strategies for improvement in addition to comprehensive school reform. The district has 
invested significantly in teacher leadership through its shared decision-making and lead teacher programs. 

Additionally, the district provides ongoing teacher coaching on literacy and aligning instruction to standards for 
schools that participate in either of these initiatives. Furthermore, the district provides mentors for new and struggling 
teachers, and hosts teacher-interns from a local university. Finally, the district contracts with an independent staff 
development agency to provide workshops and training for teachers on topics that the district identifies as important for 
instructional improvement. Cincinnati's initiatives have made significant district resources available to school sites 
undertaking reform, and have focused schools' efforts on raising student achievement to the district's standards. 

In the 2000-2001 school year. Elm Street School had approximately 400 students in grades K-8, almost all of whom 
qualified for free and reduced lunch; 20% of the students participated in special education. Additionally, the student 
body was highly mobile, with about a 50% student turnover rate during the course of the school year. In 1998, Elm 
Street chose to implement Expeditionary Learning Outward Bound [ELOB] to improve student performance, even 
though it was not a redesign school. ELOB is a comprehensive school reform design that utilizes authentic, integrated 
instructional units to support students’ academic skills and personal growth. For the past three years, ELOB has 
provided significant professional development for the staff so that they can meet the goals of the model. 



In addition to ELOB, Elm Street has created a multi-faceted literacy model with the assistance of an outside 
consultant. The model was designed to mesh with ELOB, and has been reviewed by ELOB consultants. It has several 
components: Two assessment programs (one of which is computer-based) to determine student reading level, with 
corresponding reading materials at each student’s level; 90-minute literacy blocks; Four instructional assistants to work 
with teachers on literacy and provide small group instruction; One-on-one reading tutoring offered by teachers and 
teacher-intems for struggling students in the six weeks leading up to state testing. 



Elm Street has also made several structural changes to provide individual attention to help students succeed in their 
academic program. The first of these changes is reduced class size. Elm Street has 25.6 lh"E teachers, creating an 
overall student to teacher ratio of less than 16:1. Class sizes are reduced in the elementary grades to the following 
levels: Kindergarten: 1 3 to 1 ; Elementary: 15-17 to 1 ; 7th and 8th grade: 24 to 1. The smaller classes in the elementary 
grades facilitates teachers’ use of developmentally appropriate practices. Additionally, teachers were organized into 
three teams (K-3, 4-6, and 7-8 grade) to facilitate collaborative work. Elm Street also used looping and multi-age 
classes so that students and teachers could spend several years working together. 



Finally, the school has changed the allocation of time throughout the school day to provide teachers with at least 75 
minutes a week of guaranteed common planning time. Students are in school from 7:30- 1 :45 daily. Under the old 
schedule, teachers were required to be in school from 7: 15-2: 15 daily; now the teacher day begins daily at 7:30. Since 
the teacher day starts 1 5 minutes later, 75 minutes of teacher contract time is accumulated each week and used for 
meetings on Wednesday afternoon, when teachers remain until 3:30. This provides teams with 75 minutes of common 
planning time each week, which were typically used for work on ELOB curriculum development. 



The principal noted that the combination of these strategies provides a "seamless" education for students. Students 
have opportunities to build close relationships with teachers, receive instruction tailored to their individual needs, and 
participate in authentic learning activities. To support this instructional program. Elm Street invested a significant 
amount of money in professional development, much of which focused on implementing ELOB, the literacy model or 
aligning instruction to standards. The next section outlines the various sources of professional development spending 
at Elm Street. 

5. Sources of Professional Development Spending at Elm Street 

Elm Street received federal, state, local and private funding. The school site had significant control over the budgets 
from some sources, whereas other available resources were controlled at the district level. Unlike earlier studies that 
focused on data from one level, this case study tracks district and school expenditures on professional development 
from federal, state, local and private sources. Given the complexity of the data, it is useful to think of two categories of 
professional development spending that provide professional development resources to Elm Street: 

1 . District spending on the infrastructure to support professional development (such as the salaries of central 
office administrators of professional development programs, clerical support, equipment, and supplies) and on 
professional development activities and programming that are provided to school staff This category can 
further be divided into two groups: 

o a. Trackable funds: some district spending on professional development can be reasonably tracked to 
the schools that receive the direct benefit of the resources. Of district spending on professional 
development, $7.4 million dollars (approximately 73%) of district spending fell into this category. 

This includes spending on district-funded coaches that work with schools on instructional 
improvement, mentors for new teachers, courses provided by the district professional development 
academy and funds earmarked for adoption of CSRD's. This article provides significant detail on this 
category of spending; 

o b. Untrackable funds: some district spending on professional development occurs in a manner such 
that it is not possible to track which schools receive the direct benefit of the resources. Of district 
spending on professional development, $2.7 million dollars (or 27%) fell into this category. It was 
not allocated to the school level for one of the following reasons: 

• Spending was designed to build individual or district-level rather than school-level capacity 
(for example district support for individuals to pursue National Board certification) and so 
could not be accurately tracked to a given school; 

• Funds were controlled by neither the district or school (for example, the contractually 
mandated, union-controlled professional development flind); 

• Spending was too fragmented to be accurately allocated to the school level; 

• While funding was allocated for professional development, the dollars had not yet been spent. 
For these reasons it was not possible to accurately allocate this district spending to the school level. 

2. The other category of professional development spending at Elm Street includes all school-initiated 
professional development activities funded from schools’ own discretionary budgets. This would include a 
school’s use of Title I money to hire a facilitator or coach to provide teachers training and support in 
implementing a CSRD, structuring planning time to provide teachers time within the school day for 
professional development, or using the school's general fund budget for materials or travel expenses for 
professional development. 



The upcoming sections explain the sources of trackable district-level and school-level professional development 

o o BEST COPY AVAILABLE 



resources at Elm Street, 



District-Level Support for Professional Development at Elm Street 

Elm Street participated in most of Cincinnati’s professional development initiatives, including literacy coaching, 
standards coaching, teacher leadership, teacher mentoring, and the teacher intern program. Individual staff members 
also took courses on various topics offered by the district. Table 3 lists the district initiatives in which Elm Street 
participated, the strategy used for allocating the cost to Elm Street, and the cost estimate of the resources Elm Street 
received. All estimates include the cost of fringe benefits where applicable. 

Table 3 

District-Level Professional Development at Elm Street 



Initiative 


Allocation Method 


District-wide 
Expenditure per 
Initiative 


Elm Street 

Resource 

Estimate 


Teacher Leadership 


Teacher stipends allocated per teacher to 
participating schools; other initiative costs 
allocated evenly across participating schools 


$1,195,963 


$25,999 


Staff Development 
Agency Courses 


Agency costs allocated proportionally across 
schools based on prior year course-taking patterns 


$942,950 


$22,960 


Standards Alignment 
Coaching 


Costs allocated to participating schools based on 
coach’s salary 


$414,348 


$15,936 


Teacher Intern 
Program 


Salary and stipend costs allocated to participating 
schools 


$219,475 


$15,190 


Literacy Coaches 


Costs allocated to participating schools based on 
coach’s salary 


$405,674 


$ 9,998 


Peer Mentoring 


Costs allocated across schools based on number of 
new hires/intervention teachers at each school 


$632,746 


$ 6,004 


Eisenhower Math and 
Science Workshops 


Allocated proportionally across schools based on 
teacher participation in courses 


$343,371 


$ 1,576 


Total District-controlled Professional Development resources at Elm Street; 


$97,663 



In sum, the district provided Elm Street's teachers with an average of $3,815 in professional development resources 
($97,663 , 25.6 teachers = $3,815). 



As previous research has shown (Miles & Hombeck, (2000); Miles, et al., (1999); Elmore & Burney (1997); Hertert, 
1997; Miller & Lord, (1994)), districts utilize multiple sources of funding for professional development. Cincinnati is 
no exception. As Table 4 shows, Cincinnati used federal programs, state and local tax revenues and private donations 
to fund professional development. 



Table 4 

Sources of District-Level Professional Development at Elm Street 



O 

ERIC 



Initiative 


Federal 


State/Local 


Private 


Total 


Teacher Leadership 




$ 


25,999 




$ 


25,999 


Staff Development Agency Courses 


$ 5,419 


$ 


17,541 




$ 


22,960 


Standards Alignment Coaching 


$ 8,653 


$ 


4,590 


$ 2,693 


$ 


15,936 


Teacher Intern Program 




$ 


15,190 




$ 


15,190 


Literacy Coaches 




$ 


9,998 




$ 


9,998 


Peer Mentoring 




$ 


6,004 




$ 


6,004 


Eisenhower Math and Science Workshops 


$ 1,576 






$ 


1,576 


Total 


$15,648 


$ 


79,322 


$ 2,693 


$ 


97,663 


Source Percentage of Total 


16% 


81% 


3% 


100% 
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As Table 4 shows, federal dollars support 16% of the district-level professional development spending at Elm Street. 
One significant source of federal funding in Cincinnati is Title I, the federal compensatory education program. The 
federal government recommends that 2.5% of Title 1 money be used for professional development. Cincinnati used 
some of their Title I money to fund staff development agency courses and standards alignment coaching at Elm Street. 
Title 11, also known as the Eisenhower Program, provides funding for professional development in math and science. 
Cincinnati used this funding to sponsor several district workshops in which individual teachers at Elm Street chose to 
participate. 



The largest source of dollars for Cincinnati's professional development initiatives were state and local funds, which 
provide 81% of the district-level professional development resources at Elm Street. Finally, the district resources used 
at Elm Street were supplemented slightly by private donations, which were added to federal, state and local funding for 
the standards initiative. 



School-Controlled Support for Professional Development at Elm Street 

At the school level. Elm Street utilizes funding from federal, state and local, and private sources for professional 
development. Elm Street was awarded $75,000 grant from the Obey- Porter program, a federal program that supports 
the implementation of comprehensive school reform designs. All Obey-Porter funds were spent on implementing 
ELOB, which is considered here to be a form of professional development. Since Obey-Porter grants are applied for 
by individual schools, this is an excellent example of how school-level actions can increase the resources available for 
professional development in the school. 



Elm Street also participated in the federal Title I program. The school received $293,150 in Title I funding for the 
2000-2001 school year, but spent none of this for professional development even though the federal guidelines 
recommended spending 2.5% for professional development. In contrast to the Obey-Porter example, this shows how 
school discretion can be used to reduce professional development spending at a school. This decision should be seen in 
light of an overall context of significant professional development expenditures at Elm Street, and district-level use of 
Title I funds for professional development. 



Cincinnati distributes funding for Literacy Today, a state program designed to improve literacy instruction. Even 
though the literacy program carries no professional development requirements. Elm Street spent $8,000 (of the $30,000 
at its disposal) to provide substitutes so teachers could participate in literacy professional development. Cincinnati also 
participated in TechNow, a state educational technology initiative. The state recommended that 30% of this money at 
each school be spent on technology professional development. The data showed that Elm Street spent all of its 
TechNow funds to purchase computers, but that teachers accessed technology training through district-sponsored 
workshops. 



State and local funds also provided Elm Street with its general fund budget, which in addition to paying staff, materials 
and operating expenses, was the largest single source of professional development funding at Elm Street. By 
reallocating teacher time within the contract week. Elm Street was able to provide $63,983 in teacher time for 
professional development without additional costs. While adding this teacher time into an estimate of Elm Street's 
expenditures on professional development significantly increases the level of general fund spending, as mentioned 
earlier in the discussion of teacher time, including this resource use in our estimate makes it possible to compare how 
different schools and districts create time for teachers to engage in professional development. For purposes of 
comparison, however, estimates of professional development spending without including teacher time within the 
regular contract are included in a later section. In addition to using time after school for professional development. 

Elm Street paid $7,770 from the school general fund to provide for substitutes to release teachers during the school day 
to participate in professional development activities. 

Elm Street also used $38,900 from its general fund to pay for travel expenses for teachers attending ELOB conferences 
and workshops. In addition. Elm Street received a $2,500 grant from a private group to pay for the costs of travel and 
materials for conferences. Table 5 shows the sources of professional development funding at the school level, 
including the cost of teacher time within the regular contract. (Tables 7 and 8 show how the cost of teacher time 
within the regular contract influences estimates of professional development costs). Professional development 
spending at Elm Street, like at the district level, comes from several sources. 

Two factors are most notable. The first is the large contribution of a single federal program, Obey-Porter, to 
professional development spending at Elm Street. Since all schools participating in CSRD's pay contractual fees. Elm 
Street would have had to either acquire additional resources from a different source or reallocate other resources in the 
school budget if it had not received that grant. 
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Table 5 

Sources of School-Level Professional Development at Elm Street 



Description 


Federal 


State/Local 


Private 


Total 
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CSRD participation 


$75,000 






$ 75,000 


Substitutes for teacher release 




$ 15,770 




$ 15,770 


Teacher time after school within the regular contract 




$ 63,983 




$ 63,983 


Travel 




$ 38,900 


$ 2,500 


$41,400 


Total 


$ 75,000 


$118,653 


$ 2,500 


$196,153 


Source Percentage of Total 


38% 


61% 


1% 


100% 



Second, Elm Street contributes an average of $7,662 per teacher of professional development resources to the district- 
level spending. This highlights the importance of conducting school-level professional development spending analyses 
in districts that have decentralized budgeting. It is worth noting that while the findings are not generalizable to other 
schools in the district, other schools in the broader study also used site discretionary funds to dramatically increase 
professional development resources for their teachers (Fermanich, 2001). 

Other Resources for Professional Development at Elm Street 

Elm Street also has other resources that support professional development that were not quantified in this study. More 
specifically, there were three major types of additional staff development resources at Elm Street that were not 
included in the analysis; 

• As mentioned earlier, 27% of Cincinnati’s professional development spending could not be tracked to the 
school level. Across the district, this is equal to an average of $1,038 per teacher. This could be added to the 
tracked spending for a sense of the total per teacher cost of Elm Street's professional development 
opportunities, as is done on the next page. 

• Additionally, since Elm Street participated in the teacher intern program with a university, teachers were 
eligible to apply for individual grants from the university to support their own course-taking. Since these are 
outside of district and school control and no records of participation were obtained, this study did not include 
them. 

• Finally, previous research (especially Little (1987), which attempted to quantify this resource) found that 
uncompensated teacher time was a significant resource for professional development. At Elm Street this 
included, among other things, graduate classes that several teachers took at a local university, and 
uncompensated collegial work on instruction outside of the contract day. However, since the neither the 
district nor the school site bears any of this cost, it was not included in the analysis. 



Leaving these activities out of the analysis potentially leads to an underestimate of professional development resources 
at Elm Street; however, accurately quantifying these resources is outside the scope of this study. Table 6 presents a 
summary table presenting total, per teacher and per student professional development resources at Elm Street at both 
the district and school level. 



Table 6 

Professional Development Resources at Elm Street by Level 



Locus of 
Control 


Percentage Spending per 
level 


Total Professional Development 
Spending 


Total per 
Teacher 


Total per 
Student 


School level 


67% 


$196,153 


$ 7,662 


$491 


District level 


33% 


$ 97,663 


$3,815 


$244 


Total 


100% 


$293,816 


$11,477 


$735 



As Table 6 shows, Cincinnati and Elm Street combined provide for $1 1,477 per teacher in professional development 
resources. If the untracked average district-level expenditure per teacher of $1,038 were included, this estimate would 
be $ 1 2,5 15 per teacher. Since we are unable to definitively track these resources to Elm Street, however, the higher 
estimate is not used in this study. 

6. Cost Structure of Professional Development Spending At Elm Street 

The remainder of this article analyzes Elm Street's professional development resources by the six-element cost 
structure discussed earlier, which provides a framework for understanding how resources were allocated within the 
school. Table 7 provides a breakdown of all district and school level professional development expenditures at Elm 
Street. If an ingredient within the cost structure is not listed. Elm Street did not have any expenditures in that category. 



Table 7 



Expenditures at Elm Street by Cost Element and Ingredient 



Cost Element 


Ingredient 


Description of expense 


Cost 


Teacher Time 


Within Regular Contract 


Time after School 


$63,983 




Outside regular contract 


Stipends 


$129 






Substitutes 


$15,770 




Total Teacher Time 




$79,882 


Training and Coaching 


Training 


Purchased Training, including the ELOB 
contract fee (with conference costs 
excluded) 


$89,290 




Coaching 


Salaries of District Coaches 


$41,299 






Purchased Coaching 


$15,936 




Total Training and 
Coaching 




$146,525 


Administration of Professional 
Development 


Administrative salaries 


Salaries for administration of district 
professional development programs 
allocated to participating schools 


$5,000 




Overhead 


Overhead costs for administering district 
professional development programs 
allocated to participating schools 


$16 




Total Administration 




$5,016 


Materials, Equipment and 
Facilities Used for Professional 
Development 


Materials 


Materials costs for district professional 
development programs allocated to 
participating schools 


$761 




Total Materials, 
Equipment and 
Facilities 




$761 


Travel and Transportation for 
Professional Development 


Travel 


Travel to ELOB conference 


$41,400 




Transportation 


Transportation costs for district 
professional development programs 
allocated to participating schools 


$132 




Total Travel and 
T ransportation 




$41,532 


Tuition and Conference Fees 


Conference Fees 


ELOB conference fees included in the 
CSRD contract 


$20, 100 




Total Tuition and 
Conference Fees 




$20,100 


Total 






$293,816 



Elm Street had two main types of expenditures for Teacher Time: $63,983 of teacher time within the contract when 
students were not present was used for team meetings after the student school day; (no planning time provided by 
specialist teachers was used for professional development); and $15,770 was used for substitutes to provide release 
time for teachers to attend professional development activities. In addition, the district spent $129 for teacher stipends 
at Elm Street, as calculated by allocating the teacher stipends expenditures across schools participating in the training 
program. Purchased training comprised almost 61% of total Training and Coaching expenditures; of the $89,290 for 
purchased training, $54,900 was for the ELOB contract fee, while the remainder covered a variety of workshops. Most 
of the coaching expenditures were for salaries of district personnel who provided coaching ($41,299), however, the 
district hired some consultants as coaches for specific professional development initiatives ($15,936). The majority of 
Administration expenditures ($5,000) were for salaries of district administrators, however one district program had 
expenditures for materials necessary to administer the program of which $16 were allocated to Elm Street. Materials 
expenditures by different initiatives were also allocated across participating schools, for a total of $761 at Elm Street. 
Elm Street's travel expenses ($41,400) were all related to attending ELOB conferences and training events; the 
expenditures for transportation ($132) that were allocated to Elm Street covered travel within the district for a 
particular coaching initiative. Finally, an estimated $20,100 of conference fees were separated from the ELOB contract 
cost to estimate Elm Street's expenditures for staff to attend professional development conferences. 



For purposes of analysis, it is useful to focus on the comparison of spending levels across cost elements, which is 
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shown in Table 8. Of Elm Street’s expenditures for professional development, 77% were for either Teacher Time or 
Training and Coaching. Elm Street had more expenditures for Training and Coaching than any other cost element, 
spending $89,290 (or 61% of Training and Coaching expenditures) for training and $57,235 (or 39% of Training and 
Coaching expenditures) for coaching. Most of the Travel and Transportation and Tuition and Conference Fees 
expenditures were for participation in ELOB. Expenditures for Administration, and Materials, Equipment and 
Facilities were only a small portion of total professional development expenditures. 

Table 8 

Resources for Professional Development by Cost Structure 



Cost Element 


Expenditure 


Average 
Expenditure per 
Teacher 


Average 

Expenditure per 
Pupil 


Percentage of Total 
Professional Development 
Expenditures 


Teacher Time 


$79,882 


$3,120 


$200 


27% 


Training & 
Coaching 


$146,525 


$5,724 


$366 


50% 


Administration 


$5,016 


$196 


$13 


2% 


Materials, Equip. & 
Facilities 


$761 


$30 


$2 


< 1% 


Travel & 
Transportation 


$41,532 


$1,622 


$104 


14% 


Tuition & 
Conference Fees 


$20,100 


$785 


$50 


7% 


Grand Total 


$293,816 


$11,477 


$735 


100% 



As noted earlier, it is important to include the cost of teachers' time within the school contract as part of a discussion of 
the cost of professional development. At Elm Street, however, this time had no additional cost. For purposes of 
comparison. Table 9 shows Elm Street’s professional development expenditures excluding teacher time within the 
regular school day. Excluding the cost of teacher time within the contract, professional development resources at Elm 
Street appear even more heavily concentrated in training and coaching. Additionally, this comparison shows the extent 
to which school-level resource allocation decisions enabled Elm Street to direct existing resources in teacher time to 
professional development without increasing spending. 



Table 9 

Resources for Professional Development by Cost Structure, Not Including Teacher Time within the 

Regular Contract 



Cost Element 


Expenditure without 
Teacher Time within 
the Contract 


Average 
Expenditure per 
Teacher 


Average 
Expenditure per 
Pupil 


Percentage of Total 
Professional Development 
Expenditures 


Teacher Time 


$15,899 


$621 


$40 


7% 


Training & 
Coaching 


$146,525 


$5,724 


$366 


64% 


Administration 


$5,016 


$196 


$13 


2% 


Materials, Equip. 
& Facilities 


$761 


$30 


$2 


< 1% 


Travel & 
Transportation 


$41,532 


$1,622 


$104 


18% 


Tuition & 
Conference Fees 


$20,100 


$785 


$50 


9% 


Grand Total 


$229,833 


$8,978 


$575 


100% 
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The professional development expenses for ongoing implementation of a comprehensive school reform initiative 
appear across four cost elements, including teacher time within the regular contract. The expenditures for travel and 
transportation and tuition and conference fees, a total of $61,632, enabled teachers to attend the ELOB conference. 
Additionally, $54,900 of the training and coaching expenditures paid the fee for participating in ELOB; this provided 
the school with consultants and other support for implementation. 
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In addition to expenditures for participation in ELOB, the training and coaching expenses covered $67,234 in district 
coaches across a variety of initiatives, of which $16,000 provide stipends for teachers to provide professional 
development to their colleagues. Workshops across a variety of topics cost $24,39 1 . Spending for training and 
coaching at Elm Street was somewhat fragmented, but significant amounts of spending were directed towards major 
school improvement strategies of comprehensive school reform, improving literacy instruction, and teaching to 
standards. 

Discussion 

The methodology of interviewing multiple central office staff as well as the principal to identify professional 
development expenses at Elm Street helped the researcher gain a more complete picture of spending than would have 
been possible from an analysis of budget data alone. This methodology provided much more detailed and accurate 
information about the district program resources available at the school site. Furthermore, this study uncovered some 
resources that were not apparent from school budget data. One example of this was the clarification of how substitute 
time is used. Through interviews, it was determined that of the $1 1,655 Elm Street spent for substitutes, $7,770 went 
to provide teachers release time for professional development. 



The most interesting finding of this study is the high level of expenditure per teacher for professional development. 

Table 10 

Previous Estimates of Professional Development Spending 



Author's Name (year) 


Spending per 
Teacher 


Spending per 
Student 


Notes 


Little, et al. (1987) 


$1,931 


$93 


Excluding the future value of credits and 
uncompensated teacher time 


Hertert(1997) 


$3,825 






Killeen, Monk and 
Plecki (2000) 




$223 


Did not report spending per teacher 


Caret (1999) 


$529 




Estimate for high quality professional 
development 


Elmore and Burney 
(1997) 


$1,427 






Miller, Lord and Dorney 
(1994) 


$2219-$4461 






Miles, etal( 1999) 


$5,170 




Estimate per teacher and principal ?incl teacher 
time? 


Miles and Hombeck 
(2000) 


$2078-$7002 




Estimate including cost of teacher time within the 
contract 


Elm Street Case 


$11,477 


$735 


Estimate including cost of teacher time within the 
contract 



Including time within the teacher contract, the district and school spent an average of $1 1,477 on resources of 
professional development; excluding this time still left an average of $8,978 per teacher. 

Table 10 above shows how these findings compare with earlier research on professional development spending. All 
dollar amounts have been reported in 2000 dollars, the year from which Elm Street data was collected. Table 10 raises 
a very important question: \\Tiy do the estimates for Elm Street's professional development expenditures appear so 
much higher than those found in other studies. There are three main reasons. 



Unlike much earlier research. Elm Street data was collected using a multi-step methodology. The researcher 
supplemented budget data, traditionally the main source of data for professional development cost studies, with 
interviews that enabled the researcher to more accurately determine which expenditures were directly related to 
professional development. Without these interviews the data would have been much less precise, since current 
accounting systems are not designed to clearly identify all types of professional development expenditures. 



Additionally, data was collected on school discretionary as well as district expenditures for professional development. 
If only the district-level spending were taken into account, the researcher would have only estimated Elm Street 
professional development expenditures of $3,814 per teacher or $244 per student. This is within the range found in 
some other studies, but clearly underestimates the total professional development resources available to Elm Street 
teachers. 



The methodology and cost framework utilized in this study includes teacher time within the regular contract in the 
estimate. This adds $2,499 to the per teacher estimate of professional development expenditures. Similarly, Miles and 
Hombeck (2000) include district-level spending for teacher time, which includes district-wide inservice days, in their 
calculations. Since Cincinnati does not include such days in the district contract, the teacher time in our estimates is all 
provided on the school level. Regardless, with teacher time removed, the estimates fall closer to those estimated in 
other studies. 

Unfortunately, beyond these three reasons it is not possible to tell exactly why the expenditure estimates vary across 
studies. There are two potential reasons: 

1. The districts studied had varying levels of investment in professional development. This possibility is 
supported by cross-district comparisons such as those by Hertert (1997), Miller, Lord and Domey (1994), and 
Miles and Hombeck (2000), which found significant variation in professional development spending across 
districts: 

2. The data upon which different researchers based their analyses included different items under the category of 
professional development. This possibility is also supported by existing research. The most notable example 
would be Killeen, Monk and Plecki (2000) who used a definition of professional development that included a 
wide range of expenses that fall outside most definitions of professional development, for example audio- 
visual supplies. 



The variation in definitions of professional development embedded in data sources will inherently continue. Unless 
researchers move beyond the use of traditional accounting codes, however, it will not be possible to disentangle 
differences in school districts' categorizations of expenses from differences in actual spending for professional 
development. Once expenditures are clearly identified by researchers, it is important to describe them using a 
comprehensive cost structure so that expenditures can be described transparently, making it feasible to compare 
findings across studies. 



The Odden, et al. (2001) cost structure helps to provide a substantial level of detail about the nature of spending being 
studied. Using this cost structure, it is possible to break out the variation caused by different definitions of professional 
development (for example inclusion of teacher time within the contract or ongoing coaching) from those caused by 
variations of spending. Use of this framework thus creates estimates that are more empirically and practically useful. 

By using the multi-level methodology to collect data and the Odden, et al. (2001) cost structure to analyze the data, it 
was possible to determine several things about professional development spending at Elm Street. 

Elm Street spent a significant amount of money on professional development: $293,816 (or SI 1 ,477 per teacher) 
including teacher time within the regular contract and $229,833 (or $8,978 per teacher) not including teacher time 
within the regular contract. This amount is much higher than typically recognized. While this finding should not be 
generalized beyond Elm Street, the use of the cost structure makes it possible to compare the findings to other studies. 
This leads to the hypothesis that the inclusion of school level spending, which provides 67% of the professional 
development spending at Elm Street (or 57% not including teacher time within the regular contract) is one of the 
reasons the estimate is higher than many. Since the school level is the source of such a significant amount of teachers' 
professional development resources, future studies interested in better understanding professional development 
resources available to teachers could benefit from a multi-level analysis. 



Elm Street utilized $146,525, or 50% of expenditures (67% not counting teacher time within the contract) for training 
and coaching. Of this, $57,235 was for non-traditional forms of staff development, like on-site, ongoing literacy 
coaching. This is significant since coaching, facilitating and on-site work with consultants are more likely to contain 
the elements of effective professional development identified by Caret, et al. (1999) and Odden, et al. (2001). This 
suggests that analyses of professional development spending that do not include such activities may miss expenditures 
that are substantial and more likely to be effective than typical one-day workshops. 



Elm Street spent very little on teacher time outside of the regular contract. This is partially indicative of the fact that 
Elm Street restructured time within the teacher contract to regularly provide time for professional development, 
interviews also indicated that teachers spend a significant amount of uncompensated time on professional development 
at Elm Street. Of the spending on teacher time, $15,770 is for substitutes to release teachers for professional 
development, while only $129 — stipends for attending an Eisenhower math and science workshop — compensated 
teachers for time outside of the school day. While this is very cost efficient in the short term, over the long-term it is 
unclear if this reliance on uncompensated teacher time will be a sustainable pattern for Elm Street. 

8. Conclusion 

This study focused on professional development spending at an urban elementary school engaged in focused reform. 
Budgets and multiple interviews were combined to form a rich data set. The methodology led to a detailed and 
accurate assessment of professional development spending at both the district and school level. The addition of the 



school-level expenditure data led to significantly higher estimates of professional development spending, which more 
accurately reflect the professional development resources available to teachers within a given school than do analyses 
that only utilize district data. The cost structure analysis yielded useful findings about not just the total professional 
development costs at Elm Street, but more importantly, the strategic allocation of resources. The cost structure also 
makes it possible to see how different definitions of professional development shape findings. It is thus an important 
contribution to the field since widespread use of such a cost structure would facilitate comparing findings across 
studies. 

This project represents an early step of a broader research agenda, which ultimately seeks to identify the level of 
professional development spending and spending strategies that will enable schools and districts to improve teaching 
and student learning. Continuing these analyses in a systematic manner and employing the cost methodology defined 
in Odden, et al. (2001) will yield more data that will add to the extant knowledge on the cost of effective professional 
development. It would also be interesting to apply this study’s methodology to schools that have been successful in 
generating significant student achievement gains to begin to look at the link between professional development 
spending and student achievement gains. 

Note 

This article was prepared for the Consortium for Policy Research in Education, Wisconsin Center for Education 
Research, University of Wisconsin-Madison. The research reported here was supported by a grant from the U.S. 
Department of Education, Office of Educational Research and Improvement, National Institute on Educational 
Governance, Finance, Policy-Making and Management, to the Consortium for Policy Research in Education (CPRE) 
and the Wisconsin Center for Education Research, School of Education, University of Wisconsin-Madison (Grant No. 
OERI-R3086A60003). The opinions expressed are those of the authors and do not necessarily reflect the view of the 
National Institute on Educational Governance, Finance, Policy-Making and Management, Office of Educational 
Research and Improvement, U.S. Department of Education, the institutional partners of CPRE, or the Wisconsin Center 
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This article is an analysis of professional development spending in a recently restructured urban 
high school. This study describes the school's restructuring effort, the ways in which professional 
development in the school supports the effort, and the ways in which the school reallocated 
resources to create funds for professional development spending. We then apply the framework 
of professional development costs proposed by Odden, Archibald, Fermanich and Gallagher 
(forthcoming) to the professional development expenditures in the school. Information regarding 
professional development expenditures was obtained from budget and planning documents as 
well as interviews with school and district personnel. These data revealed that teachers in this 
school on average received $9,71 1 of professional development resources with 98% of the 
spending on teacher time and training or coaching. 



Districts and schools around the country are being called upon to improve the performance of their students. In an 
attempt to do so, they are implementing whole school designs, reducing class sizes, adopting new curriculums and 
increasing both the quantity and the quality of professional development for teachers. Experts have come to believe 
that this focus of effort and resources on professional development is necessary to reach the high student achievement 
goals set by standards-based reform (Birman et al., 2000; Corcoran, 1995; Hertert, 1997; Killeen, Monk and Plecki, 
2000; Little, 1993). With this in mind, CPRE researchers at the University of Wisconsin-Madison embarked on a 
study of the costs of effective professional development. This study involved a couple of different stages. We began 
with a review of the literature on effective professional development and its costs, which held a lot of information on 
what constitutes effective professional development but very little on its costs. Furthermore, the extant research on 
professional development costs lacked a common framework for analyzing and discussing the costs at the state, 
district, and school level. This led the CPRE researchers to develop a new framework for assessing the costs of 
professional development that might become a common framework (Odden, Archibald, Fermanich, and Gallagher, 
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2001 ). 



At the same time, CPRE researchers Fermanich and Gallagher began collaborating with Karen Hawley Miles on a 
study of district and school level professional development expenditures in Cincinnati. Miles had already done 
similar work at the district level in three other districts: San Antonio, Boston and Albuquerque. The work in 
Cincinnati goes beyond an analysis of expenditures at the district level and evaluates school-level professional 
development spending as well. This project's first case study, written by Alix Gallagher (2001) about a Cincinnati K- 
8 school, was the first to systematically estimate costs at the school level and use the Odden et al. (2001) framework 
to analyze the professional development resources at a school engaged in comprehensive school reform. This case 
study uses Gallagher's format, and focuses on the different kinds of professional development now being employed by 
a restructured high school in Cincinnati and the costs of that professional development. 



This case highlights Harrison Place High School (Note I), a magnet school that used professional development as an 
integral part of a coordinated school wide reform. Professional development is defined here as organized district and 
school activities intended to build teacher knowledge and skills. This includes activities that are often classified as 
professional activities to promote instructional improvement; for example, workshops, teacher coaching, work with 
consultants, and collaborative planning time and in-service days used for activities designed to lead to professional 
growth. Since this case study examines resource use rather than economic costs, it does not cover activities that have 
no cost for the district or school, including uncompensated teacher time. This case study seeks to answer three 
questions: 1) What is Harrison's strategy for improving student achievement? 2) How does professional development 
support this strategy? 3) How does Harrison allocate resources for professional development? 



In an attempt to answer these questions, we developed a framework, collected data from documents and interviews, 
and aligned that data with our framework. These steps are described in the next section. This article does not involve 
an extensive literature review. See previous work by Gallagher (2001) and Fermanich (2001), for an extensive review 
of research on professional development costs, including an analysis of the strengths and weaknesses of those studies. 
Following the section on methodology, Harrison's educational strategy is explained in light of its district context. The 
final section examines Harrison's professional development spending in terms of funding sources and the Odden et al. 
(2001) cost structure. 

2. Methodology 

Based on prior research, the following goals were set for the study: a) to develop a methodology based on prior 
research that provides a good estimate of professional development expenditures at the school site and information on 
the nature of professional development activities;and b) to use a systematic framework for analyzing professional 
development expenditures. 



This case study builds on Elmore and Burney's (1997), Miller, Lord and Domey's (1994), and Miles’ (Miles et al., 
1999; Miles and Hombeck, 2000; Miles, 2001) methodologies, but takes them a step further by analyzing 
expenditures in terms of a clearly articulated cost structure and tracking district spending on professional development 
to the school site. 



Our first step was to identify a more specific framework for analyzing professional development costs. In a review of 
literature on the costs of professional development, Odden et al. (2001) built on the Garet et al. (1999) and Elmore 
and Burney (1997) studies (among others) to create a systematic framework for analyzing the costs of professional 
development. They then collaborated with Jennifer King Rice of the Finance Project to create a common framework 
consisting of six core and two optional (Note 2) elements to be used to analyze the costs of professional development. 
The six core elements are: 1) teacher time, 2) training and coaching, 3) administration, 4) materials, equipment and 
facilities, 5) travel and transportation, and 6) tuition and conference fees. Table 1 depicts this cost structure. 

Table 1* 

A Cost Structure for Professional Development 



Cost Element 


Ingredient 


How Cost is Calculated 


Teacher Time Used for 


Time within the regular 




Professional 


contract: 




Development 


1 . -when students are not present 
before or after school or on 
scheduled in-service days, half 
days or early release days 


1 . teachers' hourly salary times the 
number of student free hours 
used for pd 






2. the cost of the portion of the 




2. -planning time 


salary of the person used to cover 
the teachers' class during 
planning time used for pd 
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3. 


Time Outside the regular 
day/year: 

-time after school, on weekends 
or for summer institutes 


3. the stipends or additional pay 
based on the hourly rate that 
teachers receive to compensate 
them for their time 




4. 


-release time provided by 
substitutes 


4. substitute wages 


Training and Coaching 




Training 






I. 


salaries for district trainers 


1 . sum of trainer salaries 




2. 


outside consultants who provide 
training; may be part of CSRD 

Coaching 


2. consultant fees or comprehensive 
school design contract fees 

3. sum of coach and facilitator 
salaries 




3. 


salaries for district coaches 
including on-site facilitators 


4. consultant fees or comprehensive 
school design contract fees 




4. 


outside consultants who provide 
coaching; may be part of CSRD 




Administration of 

Professional 

Development 


Salaries for district or school level 
administrators of professional 
development programs 


salary for administrators times the 
proportion of their time spent 
administering pd programs 


Materials, Equipment 
and Facilities Used for 
Professional 
Development 


1. 

2. 


Materials 

Equipment 


I . materials for pd, including the 
cost of classroom materials 
required for CSRDs 




3. 


Facilities 


2. equipment needed for pd 
activities 








3. rental or other costs for facilities 
used for professional 
development 


Travel and 
Transportation for 
Professional 
Development 


1, 

2. 


Travel 

Transportation 


1 . Costs of travel to off-site pd 
activities 

2. Costs of transportation within the 
district for professional 
development 


Tuition and Conference 
Fees 


1. 

2. 


Tuition 

Conference Fees 


1. Tuition payments or 

reimbursement for university- 
based pd 








2. Fees for conferences related to 
pd 



♦Reprinted from Odden, Archibald, Fermanich and Gallagher, A cost framework for professional development. 
Journal of Education Finance, forthcoming. 



These cost elements provide a meaningful level of detail on how money is spent for professional development at the 
district and school. The usefulness of this sort of framework for making comparisons across studies becomes most 
apparent when analyzing the studies by Miller, Lord and Domey (1994), Miles, et al. (1999) and Miles and Hombeck 
(2000). Although these studies use somewhat similar methodologies, it is difficult to draw conclusions across studies 
about the level and effectiveness of professional development spending without a shared analytic framework. We 
hope our comprehensive framework, which was developed in cooperation with other professional development 
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researchers and has now been employed in three case studies, offers a useful standardized framework to allow cross- 
site analyses. 

As previously mentioned, the data collection for this study began in conjunction with Miles' (2001) multi-district 
analysis of expenditures on instructional improvement. First, data were collected on instructional and school support 
at the district level. As in Miles' earlier work, instructional and school support were defined as all district supports for 
high-quality instruction, including professional development. The analysis began with the entire district general fund 
budget, as well as those from all other public and private sources of funding for the district. The analysis proceeded 
according to the following six steps: 



• a) District-level interviews were used to develop an understanding of which expenditures were related to 
instructional and school support and to code spending in all departments within the various categories of 
instructional support: professional development, accountability, curriculum development and support, special 
program monitoring and compliance, information systems, district student services and community outreach. 
Interviews were conducted with the people in charge of departmental and categorical budgets. The 
interviews provided data on which district initiatives supported instructional and school support, the type of 
spending each related line item represented, and the percentage of salary costs for relevant individuals that 
should be considered instructional and school support, 

• b) At that point, the focus narrowed to those expenditures within instructional and school support that we 
defined as professional development. This included, for example, district literacy coaches, stipends paid by 
the district for lead teachers, the costs of comprehensive school reform design contracts, salary costs for those 
coordinating professional development, consultant fees, materials costs, and the district's professional 
development center. The analysis includes an estimate of the cost of teacher planning time that was actually 
devoted to professional development. However, the cost of salary advancements due to professional 
development credits and the cost of uncompensated teacher time were not included. 



• c) For each line item, several types of data were collected: the description, source, control (e.g. district, 
school), type (e.g. consultant fee, stipends), topic (e.g. literacy, standards), and form of delivery (e.g. school- 
based coaching, workshop). With this level of detail, it was possible to sort data according to general 
initiative (e.g. literacy, standards, teacher leadership) as well as by cost element (teacher time, trainers and 
coaches, other costs), 

• d) Professional development costs from all district budgets were then allocated, where possible, to the school 
level. The cost of each initiative was allocated among participating schools based on the staff and overhead 
costs in one of three ways: 



o By participating school — for example, if twenty schools participated in a literacy program, the 

overhead costs for the entire coaching program would be split evenly amongst the twenty schools. If, 
within that program, five schools shared a literacy coach, each school was 'charged' for 1/5 of that 
coach's salary. 

o By pupil at each participating school — for example, one initiative provided a block grant to 

participating schools based on student enrollment. The number of pupils at each participating school 
was multiplied by the per pupil funding formula to determine the resource level at the school; 
o By participating teacher — for example if the district offered an after school workshop that cost 
$1 ,000 to produce and five teachers attended, each of their schools would be 'charged' $200. 

A later section explains the precise method of calculating the cost for each initiative that involves 
Harrison Place High School. 



• e) Once district-level spending was tracked to the schools, resources from the school site budget and 

categorical programs were analyzed. Line item budgets were available for both the school general fund and 
Title I (the federal grant that provides supplemental educational funding for low-income students) budgets. 

For Title VI (the federal class size reduction initiative), Obey-Porter (the federal comprehensive school 
reform demonstration project). Literacy Today (a state literacy initiative) and TechNow (a state technology 
initiative) only total allocations were available. All of these sources were added to the district information to 
generate a preliminary estimate of resources controlled by the school. The next step ascertained how many of 
these resources were used for professional development. 



• f) Using the data collected in the earlier steps, preliminary and follow-up phone interviews were conducted 
with the principal to verify information, identify how the school used categorical dollars and its discretionary 
control to increase or reduce professional development resources at the school site. The interviews also 
provided an understanding of the school's educational strategy and how resources were deployed to achieve 
school goals. At all stages, data were gathered by cost structure elements. 



O 
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These data provided three of information: 1) Qualitative information on the school's goals and strategies; 
2) A comprehensive resource use picture; 3) Descriptive data that enabled this analysis to move beyond accounting 
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codes to an understanding of the professional development strategies and their cost. 

These three types of data together made it possible to present professional development spending at Harrison utilizing 
the cost structure developed by Odden et al. (2001). Additionally, as becomes apparent in the next section, these data 
allow for a detailed explanation of exactly how the cost estimates were developed. Finally, this methodology and cost 
structure makes it possible to overcome the barriers typical accounting practices create to understanding the 
relationship between professional development and educational strategies. 

3. District And School Context 

Harrison Place High School is located in the Cincinnati Public School District, a moderately large, urban district in 
the Midwest that serves approximately 48,000 students. In the mid 1 990’s the district had relatively low 
achievement, but has made a significant attempt to generate improvements by focusing on school accountability, 
coordinated reform, teacher leadership, and instructional improvement. Cincinnati's accountability system categorizes 
schools into five performance categories, the lowest of which can trigger 'redesign,' the district's school reconstitution 
plan. Redesign schools receive a new principal and a substantially new staff, who are required to implement the 
comprehensive school reform model chosen by the school's redesign committee (comprised of four members chosen 
by the district and four members chosen by the union). Cincinnati also operates a number of magnet schools funded at 
a slightly higher amount per pupil, and supports the adoption and implementation of comprehensive school reform 
models in many schools in the district that are not redesign schools. 



Cincinnati has several other strategies for improvement in addition to comprehensive school reform. The district 
invested significantly in teacher leadership through its team leader and lead teacher programs. The district also 
provides ongoing teacher coaching on literacy and aligning instruction to standards for schools that participate in 
either of these initiatives. Furthermore, the district provides mentors for new and struggling teachers, and hosts 
teacher-interns from a local university. Finally, the district contracts with an independent staff development academy 
to provide workshops and training for teachers on topics that the district identifies as important for instructional 
improvement. Cincinnati's initiatives have made significant district resources available to school sites undertaking 
reform, and in several cases, have focused efforts on raising student achievement to the district’s standards. 

School Background Information 

Harrison Place High School is one of five schools in the Cincinnati Public School District that serves students in 
grades 9-12. Its student population is 85% African American and 15% white. Approximately 50% of students 
qualify for free or reduced-price lunch. Compared to other high schools in CPS, Harrison serves a relatively high 
number of special education students; approximately 15% of students have Individual Education Programs (lEPs). 
Table 2 provides other relevant facts about Harrison High School. 

Table 2 

Facts about Harrison Place High School 



Student Enrollment 


1550 


Grades 


9-12 


Student-teacher ratio, overall 


12:1 


Student-teacher ratio, academic classes 


17:1 


Number of special education teachers 


19 


Comprehensive School Reform Designs 


Paideia, Co-nect 



After the 1988-1989 school year, the district closed Harrison Place High School due to persistently low academic 
performance. After reopening in Fall 1989, the school was structured to create smaller, self-contained academic 
programs for students. As of the 2000-2001 school year, the school offered five academic programs that are close to 
being self-contained. In this structure, students choose one of the five programs, and then take all of their academic 
courses with only those students and teachers in their chosen program, thus creating a smaller community for both 
students and teachers. The only exceptions are for non-core academic courses such as physical education and music. 
Two of the programs students select from are national whole school designs, Paideia and Co-nect, and the other three 
are “homegrown” designs created at the school or district level. Although only one of the programs officially uses 
Paideia, the whole school considers itself a Paideia school, adopting the Paideia mission of producing graduates who 
will become lifelong learners, responsible citizens and productive workers. The five programs include: 

• Paideia: With 400 students, this program uses teachers as coaches, runs classes as seminars, and places an 
^ extra emphasis on English. 

ERIC 
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• Cincinnati Academy of Math and Science (CAMAS): With 375 students, this program helps students acquire 
specific skills unique to science and mathematics by concentrating on these areas in other disciplines. There is 
also a small program within CAMAS called the ZOO Academy. Students in this program attend classes at 
the Cincinnati Zoo and study a zoological-centered curriculum. 

• Teaching and Technology: With 250 students, this program uses the Co-nect school design, which 
emphasizes project-based learning supported by technology. 



• Communications: With 250 students, this program offers a complete high school academic studies program 
and provides students with experiences in journalism, graphics and photography, public relations, advertising, 
and technical writing. 



• Health Professions: With 225 students, this program offers the essential building blocks for any health career 
- academic coursework combined with lab and field experience in health-related or medical areas. 



In addition to five separate academic programs, Harrison made a number of changes to meet the constant need for 
teachers to engage in a wide variety of professional development. Five on-site instructional facilitators were hired, 
one for each academic program, to provide teachers with full-time support specific to their program. In 1 997-1998, 
Harrison Place also became a team-based school. Teams of teachers and students created smaller learning 
communities for students and reduced student loads for teachers. The school also began participating in the district’s 
team leader and lead teacher programs. 



Another big change at the school came in 1999 when the school changed to flexible block scheduling. This change 
allowed the principal to rearrange the schedule so that all teachers had common planning time with their core 
academic team members. The school leadership recognized that common planning time was necessary for teacher 
teams to engage in job-embedded professional development during the regular workday. By the 2001-2002 school 
year, all core academic teachers had two 45-minute planning periods per day, or 450 minutes per week. In most 
cases, teachers used one of these planning periods to meet with their team for professional development purposes 
while the other was used for personal planning time. The collaboration time allows teachers to meet and discuss 
teaching strategies, plan a curriculum unit, or meet with their instructional facilitator during the regular school day, 
and is therefore included in our cost estimate of professional development. The next section outlines the various 
sources of professional development spending at Harrison. (Note 3) 



4. Sources and Control of Professional Development Spending at Harrison 

In the 2000-2001 school year, Harrison received federal, state and local funding. While some resources are controlled 
at the district level, the school site has significant control over the budgets from most sources. Unlike previous 
studies that focused on data from the district level, this case study tracks district and school expenditures on 
professional development from federal, state and local sources. Because these data are complex, it may be useful to 
think of two categories of professional development spending at Harrison: 

1. District spending on the infrastmcture to support professional development (such as the salaries of central 
office administrators of professional development programs, clerical support, equipment, and supplies) and 
on professional development activities and programs that are provided to school staff This category can be 
further divided into two groups: 



o a. Trackable funds: some district spending on professional development can be reasonably tracked to 
the schools that receive the direct benefit of the resources. Of district spending on professional 
development, $7.4 million dollars (approximately 73%) of district spending fell into this tracking 
category. This includes spending on district-funded coaches that work with schools on instructional 
improvement, mentors for new teachers, courses provided by the district professional development 
academy and funds earmarked for adoption of CSRDs. This article provides significant detail on this 
category of spending; 

o b. Untrackable funds: some district spending on professional development occurs in a manner such 
that it is not possible to track which schools receive the direct benefit of the resources. Of district 
spending on professional development, $2.7 million dollars (or 27%) fell into this category. It was 
not allocated to the school level for one of the following reasons: 



■ Spending was designed to build individual or district-level, rather than school-level, capacity 
(for example district support for individuals to pursue National Board certification) and so 
could not be accurately tracked to a given school; 



■ Funds were controlled by neither the district or school (for example, the contractually 
mandated, union-controlled professional development fund); 
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■ While funding was allocated for professional development, the dollars had not yet been 
spent. 

For these reasons it was not possible to accurately allocate this district spending to the school level, but 
because it is such a small portion of the total expenditures, we do not believe it is a problem for our analysis. 

2. Spending for school-initiated professional development activities funded from schools' own discretionary 
budgets. This would include a school's use of Title I money to hire a facilitator or coach to provide teachers 
training and support in implementing a CSRD, structuring planning time to provide teachers time within the 
school day for professional development, or using the school's general fund budget for materials or travel 
expenses for professional development. 

The upcoming sections explain the sources of trackable district-level and school-lcvcl professional development 
resources at Harrison. 

District-Level Support for Professional Development at Harrison 

Harrison participated in many of Cincinnati's professional development initiatives, including team leaders, lead 
teachers, teacher mentoring, the teacher intern program, and math and science workshops. Individual staff members 
also took courses on various topics offered by the district through its independent professional development academy. 
While most initiatives were funded through the district general fund, funding for the math and science workshops 
came from Title II of the federal Elementary and Secondary Education Act, also known as the Eisenhower Program. 
Table 3 lists the district initiatives in which Harrison participated, the strategy used for allocating the cost to Harrison, 
and the cost estimate of the resources Harrison received. All estimates include the cost of fringe benefits where 
applicable. 



Table 3 

District Professional Development Initiatives at Harrison 



Initiative 


Allocation Method 


District-wide 
Expenditure per 
Initiative 


Harrison 

Resource 

Estimate 


Team-based schools 


Teacher stipends allocated per team leader to 
participating schools; other initiative costs 
allocated evenly across participating schools 


$1,192,959 


$85,999 


Lead Teachers 


Teacher stipends 


$587,500 


$57,000 


Staff Development Agency 
Courses 


Allocated proportionally across schools based 
on prior year course-taking patterns 


$867,134 


$11,897 


Teacher Intern Program 


By salary and stipend costs for participating 
schools 


$219,474 


$54,677 


Peer Mentoring 


Allocated across schools based on number of 
new hires/intervention teachers at each school 


$602,731 


$ 32,018 


Eisenhower Math and 
Science 


Allocated proportionally across schools based 
on current year workshop enrollment. 


$343,371 


$5,404 


Total District-controlled 
professional development 
resources at Harrison 




$3,813,169 


$246,995 



In sum, the district provided Harrison's teachers with an average of $1930 (Note 4) in professional development 
resources. 



As previous research has shown (Miles & Hombeck, (2000); Miles, et. al., (1999); Elmore & Burney (1997); Hertert, 
1997; Miller & Lord, (1994)), districts utilize multiple sources of funding for professional development. Cincinnati is 
no exception. As Table 4 shows, Cincinnati used federal programs, state and local tax revenues to fund professional 
development. 




Table 4 

Sources of District-Level Professional Development at Harrison 
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Team-based schools 




$85,999 




$85,999 


Staff Development Agency Courses 


$2,808 


$9,089 




$11,897 


Lead Teachers 




$57,000 




$57,000 


Teacher Intern Program 




$54,677 




$54,677 


Peer Mentoring 




$32,018 




$32,018 


Eisenhower Math and Science Workshops 


$ 5,404 






$5,404 


Total 


S8,212 


$241,591 


$0 


$246,995 


Source Percentage of Total 


3% 


97% 


0% 


100% 



As Tabic 4 illustrates, state and local funds represent the largest source of dollars for Cincinnati's professional 
development initiatives. These sources provide 98% of the district-level professional development resources at 
Harrison. 



In addition to the $246,995 that the district spends to provide Harrison's teachers with professional development, the 
school spends some of its site budget on professional development. These expenditures are detailed in the next 
section. 

School-Controlled Support for Professional Development at Harrison 

At the school level, Harrison spent a total of $995,986 on professional development. Most of these funds came from 
the school’s general fund budget. The only other source of school-controlled funds used for professional development 
at Harrison was a $300,000 grant from TechNow, a state educational technology initiative. The state recommended 
that 30% of the grant money at each school be spent on technology professional development. Harrison used 
approximately $15,000, or 5% for professional development at the school site; the money was spent to provide 
technology training for teachers. 



Harrison’s commitment to funding teachers for core academics means that some common school-site expenditures on 
professional development are not a part of its budget. For example, Harrison has no allocation in its budget for 
substitutes to free teachers to attend professional development activities. Instead, teachers must rely on members of 
their team to cover their classes or money from the union’s professional development fund. However, the fact that its 
five separate academic programs each have a full-time, on-site instructional facilitator means that the school spends a 
significant amount on what we term professional development in this study, even when the cost of providing common 
planning time is excluded. 



Harrison spent $980,986 from its school general fund for professional development. Most of this spending was used 
for the salary costs of teachers who provided classroom teachers with common planning time. While including 
planning time dramatically increases the estimate of professional development spending, it is included here because 
planning time provides teachers student-free time with the opportunity to collaborate with other teachers and build 
their skills. Not all schools have arranged their schedules to provide teachers with common planning time, and those 
that have are, in effect, building an infrastructure for job-embedded professional development. Although we cannot 
be sure that all of this time is used for professional development, we believe that thinking of this time as a 
professional development resource could help to justify creating joint planning time for staff, which is an excellent 
opportunity for collaboration around student learning. Furthermore, interview data revealed that common planning 
time at Harrison is used for meetings with instructional facilitators and collegial discussions about better teaching 
strategies and curriculum units. These activities clearly fall within our definition of professional development. For 
purposes of comparison, however, estimates of professional development spending without planning time are 
included in Table 5. 




In addition to providing planning time, the general fund budget was used to pay the salaries of the five instructional 
facilitators, one for each separate academic program. These full-time, on-site facilitators are licensed teachers who 
have extensive knowledge of their specific academic program. Their on-site coaching and support of teachers is an 
example of the kind of professional development that has been proven effective - ongoing, job-embedded, and 
focused on the content of the academic program. 

The additional $995,986 ($7,781 /teacher) spent on professional development at the school site underscores the 
importance of conducting school-level professional development spending analyses in districts that have decentralized 
budgeting, since the school’s discretion can both augment and detract from predicted expenditures on professional 
development. Table 5 lists Harrison’s school-controlled expenditures on professional development, which are all 
funded by state and local sources. The table also shows the percentage of the total spent on each item. 

Table 5 

Sources of School-Level Professional Development at Harrison 
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Description 


State/Local 


Percentage of total* 


Salaries (& benefits) of on-site facilitators 


$349,910 


35% 


On-site technology training 


$15,000 


2% 


Common planning time used for pd 


$631,076 


63% 


Total with planning time 


$995,986 


100% 


Total without planning time 


$364,910 


37% 



*Rounded to the nearest percent 

Professional Development Resources from Other Sources 

In addition to the public funds listed above, Harrison also has other resources that support professional development 
that were not quantified in this study. More specifically, there were two major types of additional staff development 
resources at Harrison that were not included in the analysis: 



External support for individual teachers' professional development — Harrison had two such sources. The first was the 
union's professional development fund, to which teachers in Cincinnati could apply for mini-grants to attend 
conferences that supported their professional growth. Second, since Harrison participated in the teacher intern 
program with a university, teachers were eligible to apply for individual grants from the university to support their 
own course-taking. Since these are outside of district and school control and no records of participation were 
obtained, this study did not include these. 

Uncompensated teacher time — Little (1987) found that uncompensated teacher time was a significant resource for 
professional development. At Harrison this included, among other things, graduate classes that several teachers took 
at a local university, and collegial work outside of the contract day. However, since the school site bears none of this 
cost, it was not included the analysis. 



Leaving these activities out of the analysis potentially leads to an underestimate of professional development 
resources at Harrison; however, accurately quantifying these resources is outside the scope of this study. 

Table 6 presents a summary table with total, per teacher and per student professional development resources at 
Harrison at both the district and school level. 



Table 6 

Professional Development Resources at Harrison by Level 



Locus of 
Control 


Percentage Spending 
per level 


Total Professional Development 
Spending 


Total per 
Teacher* 


Total per 
Student 


School level 


80% 


$995,986 


$7,781 


$642 


District level 


20% 


$246,995 


$1,930 


$159 


Total 


100% 


$1,242,981 


$9,711 


$801 



*This table uses all professional teaching staff, including core academic, specials, vocational education and special 
education teachers in the per teacher calculation. 



As Table 6 shows, Cincinnati and Harrison combined provide for $9,71 1 per teacher in professional development 
resources. If the untracked average district-level expenditure per teacher of $1 ,038 were included, this estimate would 
be $10,749 per teacher. Since we are unable to definitively track these resources to Harrison, however, the higher 
estimate is not used in this study. 



5. Cost Structure of Professional Development Spending At Harrison 



The remainder of this article analyzes Harrison's professional development resources by cost structure, which 
provides a framework for understanding how resources were allocated within the school. Of Harrison's $1,242,981 
total expenditures for professional development (including both district and school level spending), 98% were for 
either teacher time or training and coaching. Teacher time was the largest expenditure, $63 1 ,5 1 9. All but $443 was 
spent on providing teachers with a common planning period during each regular school day that could be used to 
engage in professional development. The remainder was used to pay teachers to attend math and science workshops 
funded by the Eisenhower program. The school spent $581 ,781 on training and coaching, spending $58,134 for 






training and $523,647 for coaching. Table 7 shows Harrison's expenditures for professional development by cost 
element. 



Table 7 

Resources for Professional Development by Cost Structure 



Cost Element 


Expenditure 


Average 
Expenditure per 
Teacher 


Average 
Expenditure per 
Pupil 


Percentage of Total 
Professional Development 
Expenditures 


Training & 
Coaching 


$581,781 


$4,545 


$375 


47% 


Teacher Time 


$631,519 


$4,934 


$407 


51% 


Travel & 
Transportation 


$704 


$6 


$♦ 


*% 


Tuition & 
Conference Fees 


$0 


$0 


$0 


0% 


Administration 


$26,049 


$203 


$17 


2% 


Materials, Equip. & 
Facilities 


$2,928 


$23 


$2 


*% 


Grand Total 


$1,242,981 


$9,71 1 


$801 


100% 



*Since this was such a small amount of money, the per-pupil and/or percentage of total expenditures calculations 
were negligible. 



As noted earlier, it is important to include the cost of teachers' time within the school contract as part of a discussion 
of the cost of professional development. While this time is included in the regular teacher contract and therefore has 
no marginal cost to the school, the school leadership had to make a number of changes to enable teachers to have such 
a substantial amount of common planning time. For this reason, we feel it is appropriate to include the estimate of the 
cost of providing the common planning time in the calculation of the total amount spent on professional 
development. However, some researchers prefer to use estimates of professional development costs that does not 
include planning time, so, for the purposes of comparison with those studies. Table 8 shows Harrison's professional 
development expenditures excluding teacher time within the regular school day. The figures are still substantial, on 
both a per teacher and a per student basis. 



Table 8 

Resources for Professional Development by Cost Structure, Not Including Teacher Time within the 

Regular Contract 



Cost Element 


Expenditure without 
Teacher Time within 
the Contract 


Average 
Expenditure per 
Teacher 


Average 
Expenditure per 
Pupil 


Percentage of Total 
Professional 
Development 
Expenditures 


Training & 
Coaching 


$581,781 


$4,545 


$375 


95% 


Teacher Time 


$443 


$3 


$* 


*% 


Travel & 
Transportation 


$704 


$6 


$♦ 


*% 


Tuition & 
Conference Fees 


$0 


$0 


$0 


0% 


Administration 


$26,049 


$203 


$17 


4% 


Materials, Equip. 
& Facilities 


$2,928 


$23 


$2 


1% 


Grand Total 


$611,905 


$4,780 


$394 


100% 



O 

ERIC 



*Since this was such a small amount of money, the per pupil and/or percentage of total expenditures calculations were 
negligible. 
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Excluding the cost of teacher time within the contract, professional development resources at Harrison are spent 
almost entirely on training and coaching. A closer look at his cost element reveals that most of it (99%) is spent on 
coaching. This cost is high for a number of reasons. First, coaching is a form of professional development with a 
longer duration than most, making it more expensive. Secondly, this school has hired five full-time instructional 
facilitators to provide coaching at the school site; their salaries account for 60% of the amount spent on coaching and 
28% of total spending on professional development at Harrison. This reflects the fact that the separate academic 
programs, including two comprehensive school designs, is one of the main strategies for school improvement at 
Harrison. The school administration's belief that a full-time, on-site coach who could meet with teachers during 
common planning time is reflected in their large investment in both facilitators and common planning time. 



While most comprehensive school designs have fees, Harrison has managed not to incur any costs from its two 
programs. Administrators and staff at Harrison agreed to let Co-NECT use Harrison as a demonstration site; in 
exchange, the school gets to use the program without paying the usual fees. Any initial fees to use the Paideia 
program were paid by the district in previous years and are not part of this analysis. 



In addition to the salaries of the five instructional facilitators, the training and coaching expenses covered $173,737 in 
coaches across a variety of district initiatives, including lead teachers, team leaders, and peer mentors. Workshops 
across a variety of topics cost $16,804, including math and science workshops funded by the federal Eisenhower 
program. The combination of district and school-level expenditures on professional development adds up to 
comprehensive, ongoing professional development for teachers at Harrison Place High School. 

7. Conclusion 

The methodology of interviewing multiple central office staff as well as the principal to identify professional 
development expenses at Harrison helped the researcher gain a more complete picture of spending than would have 
been possible from an analysis of budget data alone. This methodology provided much more detailed and accurate 
information about the district program resources available at the school site. Furthermore, this study uncovered some 
resources that were not apparent from school budget data. One example of this was the $15,000 in grant money used 
for technology training. 



This study has two very interesting findings: the relatively high estimate of the amount spent on professional 
development and how that money was spent. The first, the high level of expenditure per teacher for professional 
development, is higher than those found in other studies mentioned earlier in the article. Including time within the 
teacher contract, Harrison teachers had an average of $9,71 1 of resources of professional development. There are 
three main reasons that this estimate is higher than those found in other studies: 



Unlike much earlier research, Harrison’s data were collected using a multi-step methodology. The researcher 
supplemented budget data, traditionally the main source of data for professional development cost studies, with 
interviews that enabled the researcher to more accurately determine which expenditures were directly related to 
professional development. Without these interviews the data would have been much less precise, since current 
accounting systems are not designed to clearly identify all types of professional development expenditures. 



Additionally, data were collected on school discretionary as well as district expenditures for professional 
development. If only the district-level spending were taken into account, the researcher would have only estimated 
Harrison professional development expenditures of $1930 per teacher or $160 per student. This is within the range 
found in some other studies, but clearly underestimates the total professional development resources available to 
Harrison Place teachers. 



The methodology and cost framework utilized in this study includes teacher time within the regular contract spent on 
professional development activities in the estimate. This adds $4,931 to the per teacher estimate of professional 
development expenditure. Similarly, Miles and Hombeck (2000) include district-level spending for teacher time, 
which includes district-wide inservice days, in their calculations. Since Cincinnati does not include such days in the 
district contract, the teacher time in our estimates is all provided on the school level. 

The variation in definitions of professional development that lead to very different estimates of costs will continue 
unless researchers decide on a comprehensive and relevant definition. This project attempted to advance this effort by 
looking across several prior works on professional development expenditures to arrive at such a definition. We then 
constructed a cost structure to aid in the collection of the same expenditures across studies. Only when studies 
include the same expenditures will we be able to say that one estimate is higher or lower than another and focus on 
why that is the case. 

As illustrated in this case study, the Odden, et. al. (2001) cost structure provides a substantial level of detail about the 
nature of spending being studied. The cost structure helps highlight the second interesting finding of this study: how 
Harrison’s professional development money was spent. The highest percentage (51%) of the money spent on 
professional development at Harrison was for common planning time for teachers - time within the regular school day 
to engage in professional development. Including this cost estimate is important because school leadership at 
Harrison purposefully reorganized the school schedule to ensure that every teacher had daily common planning time 
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with his or her teacher team. The second largest percentage (47%) of the expenditure for professional development at 
Harrison was for training and coaching, and the highest portion of these funds was spent on the salaries of five full- 
time, on-site instructional facilitators. Again, this school-based expenditure is important to include because it 
represents such a substantial investment in ongoing teacher learning. 

As these examples show, using this cost structure makes it possible to break out the variation caused by different 
definitions of professional development (for example inclusion of teacher time within the contract or ongoing 
coaching) from those caused by variations of spending. Use of this framework thus creates estimates that more 
empirically and practically useful. 



In sum, the cost structure analysis yielded useful findings about not just the total professional development costs at 
Harrison, but more importantly, the strategic allocation of resources. The cost structure also makes it possible to see 
how different definitions of professional development shape findings. It is thus an important contribution to the field 
since widespread use of such a cost structure would facilitate comparing findings across studies. 

This case study represents one of the early steps of a broader research agenda, which ultimately seeks to identify the 
level of professional development spending and spending strategies that will best enable schools and districts to 
improve teaching and student learning. Continuing these analyses in a systematic manner and employing the cost 
methodology defined in Odden, et. al. (2001) will yield more data that will add to the extant knowledge on the cost of 
effective professional development. This knowledge will help districts and schools make informed decisions about 
spending on professional development. 

Notes 

This article was prepared for the Consortium for Policy Research in Education, Wisconsin Center for Education 
Research, University of Wisconsin-Madison. It borrows heavily from two other CPRE papers. The literature review, 
methodology, and general format were taken from Elm Street School: A Case Study of Professional Development 
Expenditures, by H. Alix Gallagher, another school in the same study of one district. Some details about the school 
itself were taken from: A Case Study of Dramatic Resource Reallocation to Improve Student Achievement: Harrison 
Place High School, by Sarah Archibald. The research reported in here was supported by a grant from the U.S. 
Department of Education, Office of Educational Research and Improvement, National Institute on Educational 
Governance, Finance, Policy-Making and Management, to the Consortium for Policy Research in Education (CPRE) 
and the Wisconsin Center for Education Research, School of Education, University of Wisconsin-Madison (Grant No. 
OER1-R3086A60003). The opinions expressed are those of the authors and do not necessarily reflect the view of the 
National Institute on Educational Governance, Finance, Policy-Making and Management, Office of Educational 
Research and Improvement, U.S. Department of Education, the institutional partners of CPRE, or the Wisconsin 
Center for Education Research. 

1 . The actual name of the high school described here is not used; Harrison Place High School is a pseudonym. 

2. The two optional elements are research and development and future salary obligations. We chose not to use them 
in this study. 

3. For more information on the changes made at the school, please see: Archibald, Sarah. 2001 . A Case Study of 
Dramatic Resource Reallocation to Boost Student Achievement: Harrison Place High School, CPRE Working Paper 
Series, SF-01-1, available online: http://www.wcer.wisc.edu/cpre/finance/research/reallocation.asp. 



4. To calculate the per-teacher estimate, all core academic teachers, specials teachers, vocational education teachers 
and special education teachers were included. If the estimate were only to include core academic teachers, the 
estimate would be $2954 per teacher. It is likely that most of these district initiatives were focused on these teachers, 
but not all. 
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Abstract 

When discussing the teacher quality gap, policy makers have tended to focus on teacher 
certification, degrees, and experience. These indicators have become key benchmarks for 
progress toward equality of educational opportunity, in part for lack of additional teacher quality 
indicators. This article turns attention to teachers’ academic skills. National data on teachers' 
entrance examination scores and college selectivity reveal substantial disparities by school 
poverty level. The findings commend attention to the gap in academic skills in the formulation of 
future policy and research on the teacher quality gap. 



The teacher quality gap has received attention from a broad spectrum of policy makers (see, e.g., American Council 
on Education, 1999; National Association of State Boards of Education, 1998; National Commission on Teaching and 
America's Future, 1996). Even federal legislators have recently proposed ways to close the gap in teacher 
qualifications between low-income and affluent children (see Stedman, 1999; Wayne, 2000, 6-7). This article reports 
on the results of an examination of national data on disparities in teachers' academic skills. Analysts have thoroughly 
documented disparities in teachers' experience, certification, and degrees, but very few studies assess differences 
using academic skills indicators such as teachers' college ratings or entrance examination scores. (Note 1) 



Opportunities to examine disparities in teachers' academic skills at the national level exist because of two national 
survey efforts, overseen by the National Center for Education Statistics. First, data from the Schools and Staffing 
Survey portray the public school teaching force as a whole. Second, data from the Baccalaureate and Beyond 
Longitudinal Study portray the contribution of a cohort of college graduates. This article presents analyses of both 
data sets. For organization, the article begins with a discussion of relevant theory. It then dedicates one section to each 
of the two national data sources. Its conclusion briefly discusses implications for policy making and for future 
research. 

Theory 

The basic premise undergirding this investigation is that students with lower quality teachers experience a 
disadvantage. Put into theoretical terms, student learning is a function of teacher quality. Theorists have 




conceptualized teacher quality as a set of specific knowledge areas and skills (see e.g., Shulman, 1987). But in order 
assess teacher quality disparities at the national level, one must rely on quality indicators that can be readily measured 
by teacher questionnaires, administered via sample surveys. Measures such as teachers' experience, certification, and 
degrees are taken to be indicators of the theorized components of teacher quality, such as pedagogical knowledge and 
content knowledge. Analysts have thoroughly documented disparities using questionnaire data containing these 
indicators (see Henke et al., 1997; Ingersoll, 1996; Lewis et al, 1999). 

Less attention has been paid to differences in what Mayer, Mullens, and Moore (2000) refer to as indicators of 
teachers' academic skills, such as teachers' college ratings or entrance examination scores. The case for such attention 
is clear: studies of student achievement gains confirm — more resolutely than for many other indicators of teacher 
quality — that students learn more from teachers with better academic skills (for reviews see Hanushek, 1997; Mayer, 
Mullens, and Moore, 2000, 5-7). But it is important to point out that the relationship between academic skills 
indicators and the conceptualized elements of teacher quality has not received attention from theorists. Why might 
students learn more from teachers with better academic skills? Teachers who read faster may acquire new content 
knowledge more quickly. Teachers with greater verbal facility may spend their preparation time more focused on 
lesson design than on deciding what exactly to say. Maybe teachers with better college entrance examination scores 
learned more in college, or maybe entrance examination scores and college selectivity correlate with the quality of 
teachers' precoll egiate education — a much longer educational experience than undergraduate education. 



The development of a formalized theory of teachers' academic skills is beyond the scope of this article. But such a 
body of theory will clearly become necessary in the future as researchers and policy makers think about policies that 
might remedy disparities in academic skills. 

Evidence from the Schools and Staffing Survey 

Description of the Data 

The 1993-94 Schools and Staffing Survey (Note 2) (SASS) encompasses several distinct surveys whose findings are 
readily linked. This investigation linked teacher quality indicators from SASS's teacher survey, the Public School 
Teacher Questionnaire, to school poverty levels obtained via the SASS Public School Questionnaire. 

The teacher survey is known for its detailed questions about teacher degrees and certification, but it also contained an 
indirect measure of academic skills. The SASS asked teachers to identify their undergraduate institutions. Peterson's 
Guides (1995) rates institutions on its Entrance Difficulty Index, a reasonable albeit rough proxy for academic skills. 
(Note 3) Ratings include 'most difficult,' 'very difficult,' 'moderately difficult,' 'minimally difficult,' and 
'noncompetitive.' 



To properly consider the relationship between these ratings and academic skills, one must ask exactly how Peterson's 
assigns ratings. In truth, Peterson's asks institutions to rate themselves. To guide responses, instruction booklets that 
accompany the Peterson's surveys specify thresholds on three optional criteria: (1) entering students' high school class 
rank, (2) entering students' college entrance examination scores, and (3) the percentage of applicants accepted. The 
ratings probably do not predict academic skills perfectly, since applicants' decisions depend on other factors as well 
(e.g., wealth). 



Another SASS measure that requires some discussion reveals the prevalence of poverty at each teacher's school. The 
SASS collects, from each sampled school, the National School Lunch Program participation rate. This measure comes 
with two notable flaws. First, although the income eligibility threshold varies for different family sizes, it does not 
account for geographic differences in the cost-of-living. Therefore lunch program eligible students in rural areas, for 
instance, may be better off than eligible students in central cities. 



A second problem with this poverty metric is that elementary school students exhibit higher lunch program 
participation rates than secondary school students — 40 percent and 28 percent respectively according to the 1993-94 
SASS (Henke et al. 1997, 16). As a result, apparent teacher quality disparities by school poverty level may actually 
represent differences between elementary and secondary teachers. However, analyses of the disparities in teacher 
academic skills that disaggregated elementary and secondary teachers revealed disparities substantially like those 
found for all teachers. 



The final source of uncertainty worth mentioning is nonresponse. Missing records, records without school poverty 
information, and records that could not be associated with a Peterson's rating together conspired to reduce the usable 
sample by 32 percent, to 37,874 teachers. On the one hand, the availability of so many observations ensures that 
sampling errors need not even be mentioned in the figures presented below; all standard errors were less than two 
percentage points. On the other hand, the assumption that respondents and nonrespondents do not differ 
systematically is somewhat risky. Thus the true gap in academic skills may be somewhat larger or smaller than 
depicted. 
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Before presenting results, two analytic issues must be addressed. The first involves what specifically to compare. 
Because SASS observes poverty at the school level — not the classroom level — all findings about disparities in teacher 
quality really denote differences between teachers at low-poverty schools and teachers at high-poverty schools. In 
other words, the selection of comparison categories is a matter of dividing up schools, not teachers. Analysts divide 
up the schools using lunch program participation rates; the rates act as category boundaries. For example, Ingersoll 
(1996) divided schools into 'low-poverty,' 'medium-poverty,' and 'high-poverty' according to whether lunch program 
participation was less than 15 percent, between 15 and 50 percent, or 50 percent or more. 



To best meet the needs of policy maker audiences, the present investigation examines disparities using three separate 
category schemes. The first designates all schools as either low-poverty or high-poverty. It divides them according to 
whether their lunch program participation rates are above or below 28 percent. That threshold, determined via an 
analysis of SASS data on schools' enrollments, divides schools such that the low- and high-poverty categories each 
enroll one half of all U.S. public school students. 



Similar computations yielded the remaining two category schemes: one that divided U.S. students into fourths, and 
another that divided them into eighths. (Note 4) The divisions remain divisions of schools, so the text refers to the 
individual categories as school poverty quar tiles and school poverty octiles. 



A second analytic note that must preface the findings involves the teacher quality variable. Even skeptics of the 
student achievement research would admit that students' opportunities are diminished when their teachers' academic 
skills fall below some minimum threshold. Therefore the analysis collapses the quality variable to focus on the 
percentage of teachers from institutions Peterson's rated either 'minimally difficult’ or 'noncompetitive.' It subsumes 
such teachers under the new analytic label, 'less selective.' (Note 5) For perspective, in 1993, these institutions 
conferred only about one fifth of all bachelor’s degrees, according to a weighted tabulation based on the Baccalaureate 
and Beyond Longitudinal Study. 



Having established the comparison categories and the quality metric, the discussion can now turn to results. Figure 1 
applies the three category schemes to the SASS teachers to investigate disparities in the percentages from less 
selective institutions. All three category schemes show disadvantages for higher poverty schools. The juxtaposition of 
the three comparisons into a single graphic shows that the comparison of halves hides some important variation, 
evident in the comparison in quartiles. But the further breakdown into octiles does little. An additional breakdown 
into sixteenths (not depicted) was also not fruitful. The quartile comparison thus properly summarizes the disparities. 
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Figure 1. The percentages of teachers from less selective institutions, compared using three school poverty 
categorizations: halves, quartiles, and octiles. Weighted tabulations from the 1993-94 Schools and Staffing 
Survey. 

Evidence from the Baccalaureate and Beyond Longitudinal Study 
Description of the Data 



The Baccalaureate and Beyond Longitudinal Study (Note 6) (B&B) differs markedly from the SASS. Rather than 
portray only teachers, B&B allows a unique look at the qualities of graduates from the college class of 1993 who had 
entered teaching by 1997. 

The full B&B sample actually includes many nonteachers as well; it is representative of the entire college class of 
1993. But B&B followed the sample members over time, and the 1994 and 1997 interviews determined which 
graduates had become teachers. 

B&B's teacher academic skills measure and school poverty measure are almost identical to those described above for 
the SASS. Respondents’ undergraduate institutions were linked to Peterson's ratings to yield a measure of academic 
skills. And the school poverty measure is again the National School Lunch Program participation rate for each 
teacher's school. (Note 7) B&B does contain one additional academic skills measure, however. B&B’s authors drew 
records from several sources to determine each graduate's college entrance examination scores. (Note 8) 



Two final, important characteristics of B&B are its response rate and the number of observations. A remarkably high 
90 percent of the original B&B sample completed the 1997 interview. However, many interviews failed to obtain the 
quality and school poverty information identified above. Thus, although the 1997 interview located some 967 
respondents who reported having entered public school teaching jobs, (Note 9) school poverty information was 
obtained for only 646 of them. And among those with school poverty information, 630 had Peterson's ratings, and 
only 530 had entrance examination scores. These problems limited the conclusions substantially, as discussed below. 

Results 

As in the analysis of SASS, some preliminary remarks are needed regarding the school poverty categories and the 
collapse of the teacher quality variables. The school poverty thresholds used in the B&B analysis were exactly those 
identified above in the SASS analysis. Unless the distribution of students across schools of different poverty levels 
changed substantially between the SASS measurement in 1993-94 and the B&B interviews in 1997, the SASS- 
derived thresholds provide a sufficient approximation. 



Like the analysis of SASS, the B&B analysis presented here collapses the quality variables to focus on the proportions 
of teachers falling below what might be considered a minimum threshold. The label 'less selective' thus retains the 
meaning established earlier. For the other quality variable, entrance examination scores, the analysis focuses on the 
percentages of teachers with bottom quartile college entrance examination scores, where bottom quartile is defined in 
reference to the examination score distribution of all class of 1993 graduates. 



Turning to the results, the weaknesses of the B&B data set constrained the conclusions such that, ultimately, 
replicating Figure 1 for B&B was not possible. Besides nonresponse, the principal barrier was the very limited 
number of observations; estimates for school poverty quartiles and octiles were simply not reliable, yielding standard 
errors as high as nine percentage points. (Note 10) Figures 2 and 3 therefore each compare teacher quality in the low- 
poverty and high-poverty halves only. 
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Substantial disparities are evident in these comparisons. In Figure 2, the proportions of teachers from less selective 
institutions were 22 percent at low-poverty and 37 percent at high-poverty schools. In Figure 3, the proportions with 
bottom quartile entrance examination scores were less disparate: 26 percent and 34 percent, respectively. This 
disparity was significant only at the .15 level, while all other differences were significant at the .05 level. 




Figure 2. The percentages of teachers from less selective institutions, compared using school poverty halves. 
Weighted tabulations from the 1993 Baccalaureate and Beyond Longitudinal Study, Second FoIIow-up. 




Figure 3. The percentages of teachers with bottom quartile entrance examination scores, compared using three 
school poverty halves. Weighted tabulations from the 1993 Baccalaureate and Beyond Longitudinal Study, 
Second FoIIow-up. 

In interpreting the B&B results, readers must remember that B&B does not portray disparities among the entire 
teaching force. It portrays disparities in the flows of teachers — particularly the flow from undergraduate institutions, 
within four years of degree receipt. Other flows also play a role in determining the quality of low-income students' 
teachers, such as teacher attrition and mobility. In some state-level analyses researchers have found that such 
departures are especially prevalent among those with relatively high academic skills (see e.g., Mumane, Singer, and 
Willett, 1989). 

Conclusion 

Implications for policy making 



When discussing the teacher quality gap, policy makers have tended to focus on certification, degrees, and 
experience. These indicators have been the benchmarks for progress toward equality of educational opportunity, in 
part for lack of other indicators. This article shows clearly that policy makers need to consider teachers' academic 
skills. 

According to the evidence presented here, an academic skills gap exists, and it is quite large. The proportions of 
teachers who graduated from institutions rated either 'minimally difficult' or 'noncompetitive' were 21 percent and 39 
percent in low- and high-poverty schools, respectively. Closing this gap would require the replacement or upgrade of 
Q about one sixth ( 1 8 percent) of the teaching force at high-poverty schools. 
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Disparities of similar size have appeared in the literature for only two other teacher quality indicators. One is graduate 
degree holding, which is of debatable importance (see Ballou and Podgursky, 1999, 2000; Darling-Hammond, 1999). 
The other is in-field degree holding, which, though probably important, cannot benchmark those teachers who lack 
subject-specific assignments (e.g., general elementary teachers). Thus academic skills indicators are a relatively 
powerful tool for understanding how far away the nation is from providing equally qualified teachers to 
schoolchildren from different income groups. 

Consideration of policy options that might affect academic skills disparities would require clearer theories about the 
role of teachers' academic skills. Some policy options discussed in the literature thus far relate to school finance 
(Figlio and Reuben, 1999), school choice (Hoxby, 2000), teacher licensure examinations (Ferguson, 1998; Gitomer, 
Latham, and Ziomek, 1999), and the use of academic skills indicators in determining eligibility for teacher 
scholarships and loan forgiveness (Wayne, 2000). 



Implications for future research 



Data quality limited the certainty of the findings; due to nonresponse and sampling error, the true gap in academic 
skills may be somewhat larger or smaller than depicted. But data quality sufficed insofar as it showed (1) that a gap 
exists and (2) that the gap is substantial. Those findings have a clear implication for future research: efforts are needed 
to bring data on academic skills up to par with data for other quality indicators. 



In the short-term, some additional investment to incorporate academic skills indicators into ongoing data collection 
efforts seems worthwhile. The cost may be substantial. Most quality indicators can be reliably measured via one or 
two items on a pencil-and-paper questionnaire for teachers (e.g., experience). The same is not true for academic skills 
measures, as additional effort is required to match entrance examination score records or to code institutional 
identifiers. Presumably costs and competing priorities explain why recent federally funded teacher surveys (e.g., 
Lewis et al., 1999) have not measured teachers' academic skills. 



Over the long-term, as academic skills indicators receive greater attention, the need to improve the indicators will 
become obvious. College selectivity ratings offer a fairly rough proxy. And if researchers could administer 
standardized tests to teachers, it is not clear that they would choose to administer college entrance examinations. Thus, 
although the disparities reported here are very real, serious thought will be required about what measures could better 
represent academic skills. 
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Notes 




Although the responsibility for errors belongs solely to the author, I would like to thank Daniel Goldhaber and Willis 
Hawley for very helpful comments on an earlier draft. I would also like to thank David Figlio for generously sharing 
the data he and his colleagues entered on college selectivity, 

1 . Disparities were evident in an analysis of very old national data, from 1 966 (Ehrenberg & Brewer, 1 995), and in 
recent data from the states of New York and Texas (Lankford, Loeb, and Wyckoff, 2002; Ferguson, 1998; Kain and 
Singleton, 1996). 

2. A variety of technical and methodological reports on the Schools and Staffing Survey are available online at 
http://www.nces.ed.gov/surveys/sass, 

3. Other researchers have completed the requisite data entry to link each institution to its rating in the 1995 edition of 
Peterson's. The use of the 1995 ratings is defensible if, as Hoxby (2000) has claimed, ratings are sufficiently stable 
over time. To better reflect teacher characteristics, each teacher would need to be linked to the ratings issued 
approximately four years before his or her college graduation. 



4. The lunch program participation thresholds for all three category schemes were computed via weighted tabulations 
of the data yielded by the SASS Public School Questionnaire, The threshold that divided them into halves was 28,37 
percent. The exact thresholds that divided U.S. students into quarters were as follows: 12,59, 28,37, and 51 ,80 
percent. And the seven thresholds that divided them into eighths were as follows: 6,35, 12,59, 19,67, 28,37, 38,58, 

5 1 .80, and 71 .80 percent. 

5. Richard Ingersoll (1996, 4-5) used a very similar rationale to justify his particular construction of out-of- field 
teaching. He opted to treat teachers as in-field even if they held only a minor related to the subject taught, and even if 
that minor was in a subject-related education field (e.g., mathematics education), 

6. A variety of technical and methodological reports on the Baccalaureate and Beyond Longitudinal Study are 
available online at http://www.nces.ed.gov/surveys/B&B. 

7. The architects of Bi&B foresaw that knowledge about the schools at which the B&B teachers taught could be 
useful. Therefore respondents who taught were asked to identify the school at which they taught, and responses were 
coded so that basic school characteristics could be obtained via the Common Core of Data — another data set created 
by the National Center for Education Statistics, 

8. Sources included records from the Educational Testing Service and higher education institutions' records of sample 
members' SAT and ACT scores. See Henke et al, (2000, 83), 



9. This total does not include teachers who first taught at private schools. It also excludes teachers who had taught 
before graduation or had received their certification more than one year before graduation. This latter group is often 
excluded in analyses of the B&B cohort's contribution to the teaching force. See Henke et al, (2000, 9), 

10. Further disaggregation of teachers into school poverty quartiles and octiles yielded estimates with standard errors 
as high as nine percentage points. Some interesting spikes occurred in the percentages of teachers with low academic 
skills in the fifth octile, and to a lesser extent in the third quartile. But given that the spikes may be largely an artifact 
of sampling error or nonresponse bias, it was judged that the best summary would be the comparisons that focus on 
the low-poverty and high-poverty halves. 
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Abstract 

In pursuing the goals of educational reform over the past several decades, educational policy 
makers have focused on teachers, administrators, and school structures as keys to higher 
educational achievement. As the would-be beneficiaries of reform, students, and their interaction 
with the educational system, have been almost entirely overlooked in the pursuit of educational 
excellence. Yet, as we argue, students are as causally central as educators in bringing about 
higher educational achievement. In what follows, we examine rational student interaction with 
the educational system and show why a large number of students have incentives to undercut the 
intent of the reforms. These are incentives created by our development of an educationally-based, 
meritocratic social and economic system. No one, apparently, is asking what exactly is in the 
reforms from the point of view of quite rational, if sometimes irresponsible, student self-interest. 

Indeed, the eduationally-based, meritocratic social and economic system may be actually forming 
student preferences guaranteed to result in educational mediocrity rather than excellence. Finally, 
we comment upon the meaning of "educational excellence" and show why the educational 
reformers' understanding of the purpose of public education — to compete in the global economic 
system — can only fail to capture it. In doing so, we point to the kinds of educational structures 
and policies that create multiple pathways to competent adulthood that do have a chance of 
bringing about the reformers' stated goal of excellence in the educational system. But these are 
structures and policies that challenge the entire conceptual framework of the current educational 
reform movement. 



There is a curious omission in the spate of educational reform movement reports, analyses, and recommendations 
over nearly decades of its existence. They have focused on teachers, the curriculum, school structure and quality, 
content and performance standards, teacher education, and the like. Yet almost no attention has been paid to the 
would-be beneficiaries of implemented and proposed educational reforms: students. The achievement level of 
American students is bemoaned and, arguably, documented in the international comparison studies all right. But 
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beyond being assigned the task of benefiting from the reforms (i.e., learning), students, their roles and activities, 
figure palely in the drive for higher educational achievement. 



Yet, as we shall argue, it is students — their goals, motivations, and conceptions of the good life — that may well prove 
to be the undoing of the educational reform movement. In other words, we might well improve the quality of teachers, 
legislate higher content and performance standards and academic requirements, and reform teacher education to the 
educational reform movement's content, and still totally fail in achieving anything close to educational excellence in 
our schools. The reason will be that there is nearly total disregard for rational student interaction with the educational 
system. Students, in quite rational pursuit of their own ends, are clearly capable of undermining the intent of the 
reforms. 



In what follows, then, we shall in Part I develop a view of what it means to be an ideal student. Here we shall raise the 
question of whether students have any duty or responsibility for acting as an ideal student would. And though we shall 
note that a general view of positive student responsibility can be justifiably defended — not merely asserted — we shall 
also show that students may be both irresponsible and rational in failing to act as ideal students. In Part II, we explore 
the distributive behavior of the educational system and the idea of a schooling*based meritocratic society. Here we 
shall argue that this distributive behavior strongly favors the development of the kinds of students — students in name 
only — who contrast strongly with a noted view of "the ideal student." In Part III, we shall develop a more refined 
typology of students as rational actors. We argue that in full rational pursuit of their own view of the good, these 
students in name only will rarely, if ever, find it in their interest to act as an "ideal student" would. In Part IV, we shall 
finally establish that the failure to take into account the various rational (and non-rational) interests of students will 
most likely undermine the reformers' intent. In concluding, we point the way to the kind of fundamental and more 
penetrating educational reforms that could lead the way to excellence in education and educational achievement. 

I 

The Ideal Student 

While the current educational reform movement has undergone successive changes in focus since the "A Nation at 
Risk" report was published (D. P.Gardner, et al., National Commission on Excellence in Education, 1983), the banner 
of "Educational Accountability" remains its enduring hallmark. Indeed, as Finn et al. (1985, pp. 194-195) noted from 
its inception, the educational reform movement gathered steam towards its current infatuation with state and national 
educational standards precisely because of the widespread perception that the educational profession had abandoned 
even the pretense of upholding educational standards, while disclaiming any responsibility for the sorry educational 
results. Thus, if educators were so derelict in their duty, school boards and, increasingly, state governors and 
legislatures and the federal government entered the scene to set things aright. Through such tools as higher 
requirements for teacher licensure and inservice performance and district and state, if not national, content and 
performance standards, the collective feet of educators could be held to the fire for meeting them. Student 
performance could then be monitored through standardized testing, NAEP, and new (more realistic, but more 
expensive) statewide performance assessments tailored to state standards. While student performance on the 
assessments, of whatever sort, remains the key item of interest to the "outcomes-oriented" reformers, educators clearly 
bear the onus of raising the scores. In the earlier phase of the reform movement, more than a few school districts 
proposed, or flirted with, policies to evaluate teachers individually on the assessment scores of their students. 
Relatively lower scores, as proposed then, could bring teacher probation or even dismissal (Rodman, 1986). The 
teacher unions and common sense, however, have generally prevailed in arguing against such unfair evaluation 
practices. While school districts and states may be devising new (and fair) ways to help suspect teachers and new 
incentives to reward teaching excellence, sanctions for poor student performance are now more often levied at entire 
schools, and even school districts (Darling-Hammond, 1995; Popham, 2001). 

Doubtless there are many mediocre and poor teachers in the United States and Canada who should never be visited 
upon a classroom (but probably no more as a percent than in other professions). Anyone spending time in schools will 
recognize this. Better teacher preparation, better conditions of employment, better professional development, and 
better procedures for identifying marginal teachers are clearly in order (especially as promoted by educators 
themselves). Yet, once more the reform movement's preoccupation with the teaching profession, as a whole, ignores a 
salient feature of education: education is far more than a linear technological process in which the teacher transforms 
"raw material" (the student) into a finished "product" (an educated person). Teaching, it is true, is a "making 
something happen" profession. And so there is a causal relationship between teaching and student learning (see 
Ericson & Ellett, 1987). But it is not the simple causal relationship of the manufacturing process so familiar to 
business leaders, legislators, and other leaders of the educational reform movement. Though understanding teaching 
as a "making something happen" profession, these leaders are too prone to infer that a lack of success in teaching 
entails either that the teaching was poor (or the teacher a failure) or else that the teacher was never trying in the first 
place. 



er|c 



Although central to the reform movement's demand forever increasing educator accountability, the inference from 
lack of teaching success to poor or derelict teaching is clearly fallacious. It simply ignores the causal role of students 
in bringing about their own learning. Students, obviously, are not raw materials awaiting only a teacher's skillful 
hands. They are an integral factor in the learning process. For even the best teaching in the world will produce no 
results if students fail to be concerned with their own learning and fail to master the tasks and activities necessary to 
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educational achievement — tasks and activities such as attending to explanations, practicing introduced skills, and 
doing homework. In other words, we are speaking of an interactive causal process in which either poor teaching or 
poor "studenting" is generally sufficient for a lack of student success. (Note 1) (Of course, extraordinary teachers and 
excellent students can overcome ineptitude and initial disinterest in the other party.) Not all, therefore, hangs on the 
activities of the teacher as the educational reform apparently assumes (given the broadsides against educators and 
silence on students). If the schools are failing, indeed, then we have to explore the equal possibility that it is not 
educators, as a whole, who are necessarily at fault; rather, might the blame be laid squarely at the feet of our young? 

Yet delicacy, rather than logic, might suggest this to be an indecent proposal. To entertain blaming the young for our 
educational situation may sound a bit like entertaining a proposal to torture the innocent. Still, it seems to be the only 
way to confront the educational reform movement with the logic of its own position. For if educators are fair game 
because of their causally central role in the learning process, then students, who are equally causally central, can 
hardly be spared similar attention. Fairness simply demands it. 



But much of this question concerning teacher and student accountability hinges on a prior issue that is also 
overlooked. Granted that teachers and students are two major interactive causal factors in student learning, can 
causation serve as a sufficient basis to ground teacher or student responsibility? In a previous paper on teacher 
accountability (Ericson & Ellett,1987), we strongly criticized the view that causation entails responsibility. For 
example, atmospheric conditions may cause lightning, but we would not hold atmospheric conditions morally, legally, 
or institutionally responsible for the lightning or its effects. Thus, moral or legal responsibility does not, in general, 
follow from being a causal factor. 

Therefore, as with teachers, in considering whether students are to be held responsible for their own learning, we 
require something beyond acknowledgement of their central causal role. We require a moral and/or legal theory that 
plausibly determines student responsibility. Here it may help to sketch an ideal of what it means to be a student. From 
there we can ask whether students have an obligation to fulfill the characteristics of that ideal. (Note 2) 



Clearly, the ideal of the student goes well beyond two well-known legal obligations required of all students: (1 ) to 
attend a legally sanctioned place of education (including "home schools") until a certain age and (2) to be non- 
disruptive. Rather, a normative view of an ideal student extends to the manner of their activities within the school and 
out. And though students engage in a variety of activities during a typical school day ranging from the classroom, to 
recess, to having lunch with friends, and on to engaging in co-curricular and extra-curricular activities. We shall 
construct, however, an "ideal" of the student that focuses selectively and primarily on purely scholastic and academic 
concerns. 



According to this ideal, it is a major aim, internal to the practice of education, to introduce the young child to the 
manifold ways that we have come to structure our experience of the world. (Note 3) Initially, this means enabling 
them to begin to master various skills common to decoding the intellectual traditions and disciplines that they will 
later confront (e.g., literacy skills and computation). As they develop and build on prior learning, they will be 
increasingly initiated into the whys and wherefores of the various forms of understanding that we have achieved over 
time. For example, either explicitly or implicitly, they will come to learn that the study of human history differs (in 
content, concepts, methods, and tests for truth) from the study of natural science. And as they come to see the 
differences and commonalities among these basic ways or conceptual schemata by which we have structured the 
world, they thereby become more competent interpreters, critics, and evaluators of them. As Paul Hirst puts it, it is 
what it means to come to have a mind in the fullest sense. (Note 4) 

But from this brief sketch of perhaps what many take to be the ultimate aim of education, we can derive an intuitive 
view of the ideal student. It is contained in such familiar expressions as "she is a real student of x!" Such expressions 
betoken a true zeal on the part of the learner to get on the inside, to master an area or subject for its own sake. It 
carries with it the idea that the learner is prepared to do whatever is necessary to achieve that critical mastery. In part 
this will mean, depending upon the subject, practicing, mastering, and engaging in exactly those activities 
Fenstermacher (1986) speaks of in "studenting:": attending to instructions and explanations carefully, reading closely, 
critically discussing thoroughly, investigating thoughtfully, questioning eagerly, practicing with an eye to proficiency, 
appraising carefully, etc., while prizing each new gain in understanding throughout. 



It is by engaging in these activities in such adverbial fashion that we can give meaning to such expressions as "she is a 
real student." They denote individuals who do not merely fill the institutional role of student. Rather, they define for 
many of us the concept of the ideal student. And, of course, it will be the best teachers who are skillful in enabling 
students in the institutional sense to become students in the ideal sense. 

One further aspect of this portrait of the ideal of the student remains to be emphasized. That is the question of 
motivation. In speaking of a student's zeal to get on the inside of a subject matter, we point to the fact that, whatever 
external utility exists in mastering it, such external utility does not exhaust the student's interest. In other words, the 
student values the learning primarily for its own sake and not merely for the sake of what it may lead to. Students in 
this sense are intrinsically motivated by the subject matter. And their preferences, commitments, and feelings come to 
be defined by the standards of excellence inherent in the discipline. (Note 5) 
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This ideal forms the basis for a typology and ranking of students. First, and foremost, is the student who comes to 
value a domain of knowledge for its own sake. Such individuals are purely intrinsically motivated. (They are also 
quite rare; we shall call them the "scholar" type.) Next, and somewhat more frequently encountered, are students who 
must often be given a specific external reason for studying a subject. These are students who are primarily motivated 
for reasons extrinsic to an intellectual discipline itself, but come to value it because it is essential to some professional 
(or career) goal. (We shall call them the "professional" type.) Now there may be those who are professionally- 
oriented, but who after time come to derive enjoyment from learning an intellectual discipline itself. (We can call 
them the "scholar-professional" type.) Finally, similar to, but truly unlike, the professional type of student are those 
who are solely motivated for reasons strongly external to an intellectual discipline itself, because learning the material 
eventually leads to what they really seek: status and wealth. For example, they may not really care about wanting to 
heal people, but they view the practice of medicine as highly lucrative. 



Now all four types may engage in the activities of studenting mentioned above and may sometimes be indiscernible to 
teachers. But it is mainly the scholar and scholar-professional types that fall under our concept of the ideal student. 

The purely externally motivated student, when concerned only with the status and wealth that formal education may 
help bring, is unaffected by the aim of education adumbrated above. He is a student in the institutional sense that may 
at times, when long-term self-interest is considered, mimic the ideal of the student. As we shall note in the next 
section, the dynamics of the educational system strongly fosters the development of this type of student. During other 
times, however, the status and wealth motivated student most closely resembles that most teacher-dreaded student 
type: the wholly unmotivated student (or "indifferent/hostile" types). When coupled with indifferent or even hostile 
students, the status and wealth seekers swell the ranks of those who are in the schools, but not of it. For together they 
have no abiding allegiance to the purpose of education itself. While we know of no complete survey estimating the 
population of each type of student (surely an important task that should be done), general experience suggests that 
alarmingly large numbers of our young fall into the educationally unmotivated category. (Note 6) For they are 
students in name only. In many schools, especially at the secondary level, educators are in a day-to-day struggle to 
simply find something to interest these nominal students. As we shall explore later, status and wealth seekers and 
indifferent/hostile students are quite capable of scuttling the most carefully worked out educational reforms and may 
serve as the overlooked factor in undermining the reform agenda. 

But prior to attending to such issues, we finally need to comment upon the matter of student responsibility. Granted, 
as we have argued, that students are causal agents of their own learning, can students be held morally or institutionally 
responsible for their own learning? Some school districts, such as Beverly Hills, California, request that all students 
sign a "student responsibility contract" that purportedly obligates students to perform the activities of students in the 
ideal sense. But, of course, this is a "contract" and "obligation" in name only. It is neither enforceable in a court of 
law, nor in the "court" of morality. So, this will not do. 



Teachers, on the other hand, because they receive remuneration for their services and because they assume a high 
moral office in a helping profession, have both a legal and moral responsibility to do all that is in their power and 
authority to ensure that students learn and are introduced to the activities of the ideal student. The educational 
reformers, like the educational profession itself, do have a legitimate interest in calling to task those in the profession 
who fall short in upholding that moral office. Parents and guardians, similarly, share in a legal and moral obligation to 
foster and encourage the development of their young to the utmost. And when parents themselves fall short, educators 
are correct in pointing out that parental negligence can be a major source of our educational ills. Though accurate 
enough, parents cannot be regulated by the state in the way that teachers can be. They fulfill the letter of the law by 
trying to ensure the regular school attendance of their children. Hence, educators tend to be the sole target of the 
reformers as a matter of politics and policy. 



But what of student responsibility for learning? As we have argued elsewhere, students can be held morally 
responsible in the context of a liberal democratic society. (Note 7) In so far as the chief purpose of education is held to 
be induction into the ideals of democratic citizenship, students have an interlocking right to education that comes with 
a duty to take it seriously (especially as students develop in rationality). (Note 8) Some rights also incur obligations, 
and civic education in a pluralistic democratic society (but not all societies) is one of these. 

But to the extent that educational reformers emphasize other ends as the chief purposes of education, such as national 
economic expansion and personal social and economic status, the message is muddied, if not vitiated. (Note 9) We 
might well think that personal self-interest would dictate attending to the activities of the ideal student. An interest, 
however, does not a moral obligation make. As policy makers, the educational reformers are on a slippery slope of 
their own devising. Self-interest, even rational prudence, may channel students in a completely different direction 
than that envisioned by them. We shall argue this in a variety of ways in the next section. Excellence in education and 
the Jeffersonian ideal of a democratic public cannot be purchased, no matter what the amount, by mercantilistic ends. 



II 



Meritocracy, The Educational System, and the 
Educational Reform Movement 




If a person comes to form certain goals, then one comes to have a certain interest in the means to reaching them. The 

76 



point is conceptual. If it is a young person's considered desire to truly explore a school subject, to get the most out of 
it, then it is in her interest to master the art and skills of studenting. To the student in the ideal sense, performing the 
acts of studenting and performing them well is always in her self-interest. And it might be thought that it is a major 
function of the educational system to encourage this interest in studenting and to see that it is spread to as many 
students as possible. There is no doubt that many educators at the classroom and school levels are striving to do 
exactly that. Bringing students along to become intrinsically motivated in a subject matter is widely held to be one of 
the highest aims of teaching and education. 

At the level of the education system as a whole, however, the story is rather different. Here, what is encouraged is not 
so much the attitudes and activities of studenting in the ideal sense as studenting in what we call the "systemic" sense. 
Here the goals of studenting in the ideal sense — the zestful pursuit of knowledge and understanding — give way to the 
goals of studenting in the systemic sense: the pursuit of grades, degrees, and careers. In the systemic sense of 
studenting, knowledge and understanding, at best, are merely means to these other goals. At worst, the true pursuit of 
knowledge and understanding is an impediment to their attainment. 



We shall now explore why the educational system at the aggregate level encourages studenting in the systemic sense, 
explore what those activities and attitudes are, and show how they work to discourage students from becoming 
students in the ideal sense. In Part III, we shall also comment on the rationality of the totally indifferent and even 
hostile student. 

The Distributive Behavior of the Educational System (Note 10) 

It is difficult to understand the emergence, development, and expansion of the American educational system without 
taking into account deeply-rooted, American cultural beliefs concerning the value of education. Jefferson long ago 
noted that a liberal education is essential to the preservation of the republic. The pioneers, who immediately 
established schools upon settling, saw in education the extension of civilization and the preservation of cultural 
tradition. And as the nation became transformed from an agrarian to an incipient and now full-blown technological 
society, the American school was viewed as more and more central to the creation of a skilled workforce. It is this 
latter view concerning the value and importance of education, of course, that primarily motivates the educational 
reform movement, concerned as it with America's position in the world economy. 



But there were other social forces at work that help explain the now nearly universal attendance and attainment of pre- 
collegiate education and rapidly expanding post-secondary education. Chief among these forces is that long- 
entrenched, almost fervent, American belief in the social and economic efficacy of education. It is a belief, or related 
collection of beliefs, that far predates the transformation of the early American agrarian economy. Writing in the 19th 
century, Alexis de Tocqueville (1835, 1969) clearly recognized this boundless faith in the power of education: 

Even the crowd can now plainly see the utility of knowledge, those who have no taste for its charms 
set store by its results and make some effort to acquire it... 

As soon as the crowd begins to take an interest in the labors of the mind, it finds out that to excel in 
some of them is a powerful aid to the acquisition of fame, power, or wealth (p. 458). 

The belief in the social and economic efficacy of education springs from 1 8th century liberal ideology that holds that 
social rewards and privileges belong not to an elite, hereditary class, but should go to those individuals of talent, 
intelligence, and industry. The ideology of America, if not the reality, has always been one of meritocracy. Thus, with 
the growth of the common school in the 19th century, it takes little imagination to understand how beliefs concerning 
the social and economic efficacy of education could be translated into a conviction that schooling pays social and 
economic dividends. Clearly, it is a conviction that could appeal to employers interested in the relatively greater 
profits an educated workforce could generate. And it could appeal to individuals who viewed schooling as a way to 
better their life chances. The transformation of the secondary school from an elite to a mass institution in the 20th 
century appears to have cemented the relationship between the social and economic efficacy of education and the 
conviction that schooling pays off socially and economically. (Note 1 1) In the later rapid expansion of higher 
education, we find ample confirmation of that expected relationship. 

To understand why, imagine a society that distributes social and economic benefits (income, status, earnings 
opportunities, etc.) on the basis of the distribution of purely educational benefits (knowledge, skills, judgment, etc.). 
Such a society is likely to be extremely inefficient. It is difficult and time-consuming to discover who knows more 
and who less. But if there were an intervening social institution that functions to evaluate individuals' relative 
possession of educational benefits, then such official testimony would straightforwardly provide the basis for a 
subsequent distribution of social and economic benefits. 

In our own society, it is through the development of certification in the educational system (by such instruments as 
grades, test scores, diplomas, and transcripts) that made possible the development of a relatively efficient meritocracy 
based on education and gave powerful confirmation to the belief in the efficacy of education. Further, it welded a 
hodgepodge of schools and colleges into a national educational system. For just as certification serves the social and 
economic system, grades, transcripts, etc. ^erve the educational system as a "medium of exchange." This medium of 
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exchange function of grades, transcripts, and diplomas is based on their rough "surrogate" (Note 12) capability to 
stand in for or represent the possession of relative levels of knowledge, skills, and judgment. The standard grade of 
’A," for example, is shorthand, a way of saying that a student has shown superior mastery of a subject (given a certain 
system level). It permits the avoidance of exhausting discussions of exactly what the student has mastered. And in 
their medium of exchange function, these surrogate educational benefits make possible communication among 
educational institutions creating easier transfer and placement policies between schools at the same level and ease and 
efficiency in selection and placement policies between schools at different levels (say, high school and college). Thus, 
surrogate educational benefits make the educational system possible. Yet they also provide the basis for linking the 
educational system as a whole with the social and economic system. 



It is not difficult, then, to understand how an actual educationally-based, meritocratic society works. Basically, it 
encompasses four distinct distributions of which only two are the educational system's own. They can be encapsulated 
as follows: 



Figure 1 
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Figure 1 can be understood as saying that the distribution of educationally relevant attributes (intelligence, tenacity, 
and choice) in the school-age population in large part gives rise to the distribution of educational benefits in that same 
population (some learn more than others). In turn, the distribution of educational benefits produce a distribution of 
surrogate educational benefits (some are evaluated more highly or get higher test scores than others). Finally, the 
distribution of social and economic benefits (some get better jobs, earn more, obtain higher status than others) is 
distributed on the basis of the relative distribution of surrogate educational benefits. 



Of these four distributions, only 2 and 3 are clearly distributed by the educational system directly. (The "genetic 
lottery" and early childhood life chances generate the distribution of educationally relevant attributes; the social and 
economic system directly distributes social and economic benefits.) But what is central to the idea an educationally- 
based meritocratic society is the notion that adult social and economic advantages should be based on the distribution 
of (surrogate) educational benefits. "On the basis," then, entails that there is more than an empirical likelihood of a 
positive relation between the two distributions. Rather, it has to do with the manner in which the adult distribution of 
social and economic benefits is socially legitimated. (Note 13) As Green et al. (1980, 1997, Ch. 6) put it, entailed is 
the following normative principle: "TTiose having a greater share of (surrogate) educational benefits merit or deserve a 
greater share of social and economic benefits." This educationally-based meritocratic principle provides a social basis 
for the way that subsequent social and economic inequalities can be regarded as justified. (Note 14) It is a principle, in 
other words, of distributive justice. It is the principle concealed in the very notion that schooling pays. It is also the 
principle responsible for prompting students to student in the systemic, rather than ideal, sense that we shall explore 
below. Moreover, it is the principle that may well prevent any real and lasting educational reform. It remains now for 
us to draw out how differing student types might rationally interact with a meritocratic educational and social system. 



Ill 

The Different Ways of 
Rationally Interacting with the System 

Up to this point in our treatment of the student and educational reform, we have not adequately considered the 
importance of a student's entire motivational and belief configuration. Although it is helpful to know that a person has 
a certain view of the good, more information is needed to explain, and in most cases justify, the student's interaction 
with the educational system. Beyond coming to know a person’s view of the good and their other, perhaps conflicting, 
goals and purposes, we also require an understanding of the relative strength of each of these if we are to understand 
what the person has good reason to do. In adopting this explanatory framework of student behavior, we are in effect 
considering students as rational agents who pursue alternative courses of action for which there are comparatively 
good reasons. Since young children generally have not developed a motivational and belief system sufficiently to be 
viewed as rational agents, we shall confine our analysis to the population of intermediate, senior high, and college 
students who are (roughly) rational agents. (We readily grant that many students at these levels are not ~ or only 
occasionally - rational believers and doers; their beliefs and behaviors often demonstrate that.) Here, we want to 
show that even when students are fully rational, the distributive dynamics of the educational system encourages 
student conduct that conflicts with the ideal of the student — and so creates conflicts between educational objectives 
and student objectives. These conflicting aims and desires present extraordinary challenges to and put constraints on 
the direction and degree of real educational reform. 



In assuming that the educational system contains some students who are rational, the power of our analysis depends 
crucially on the conception of "rationality" that we use. It is difficult to find answers to questions about whether an 
action is rational (reasonable), and in many areas such answers are controversial as well. (For example, it is extremely 
controversial whether every rational actor has compelling reasons to treat all people morally.) Regardless of the 
difficulty in all cases, our arguments will assume three major claims about rational agency that are well-established 
and accepted by most current philosophical research (and even here there is controversy — followers of David Hume 
would not fully endorse our second claim): 

• Cl . A rational agent need not (but can) have a dominant concern for his or her own long term welfare. 

• C2. A rational agent’s desires and beliefs are open to critical appraisal in light of the facts and logic. 

• C3. In so far as a rational agent seeks a specific goal, the agent will seek out effective means for fulfilling that 
goal. 



The first claim rejects the view that a rational person is necessarily prudent (e.g., see Parfit, 1986, on critical present 
aim views of rationality). The second claim rejects the view that only the means to fulfilling one’s desires — and not 
the desires themselves — are open to critical appraisal. In what follows, we assume, therefore, that the rational agent's 
beliefs and motivations could be appraised as reasonable or unreasonable in light of his or her circumstances. The 
third claim reminds us that all rational agents will take care to find ways to reach their goals. It is, however, beyond 
the scope and limits of this paper to provide the detailed arguments to sustain these claims. 

What reasons, then, might a student who has various capacities, beliefs, purposes, loyalties, and commitments have 
for interacting in certain ways with teachers, administrators, counselors, etc. in the system? Let us begin by 
reconsidering the ideal student who intrinsically values knowledge and understanding (i.e., for its own sake) and who 
is intrinsically motivated in the educational system. When this type of student interacts with the system, the love of 
scholastic and academic learning dominates and primarily influences the person's behavior. Of course, if basic 
economic needs are not being met apart from activities in the educational system, rationality requires that such a 
student temper her pursuit of the intrinsic benefits of learning with more career-related aims. (It may well be irrational 
for a person to go through primary and secondary school loving learning but never pondering how they are going to 
live after the close of their school years.) But still the ideal student is only motivated in minor ways by the extrinsic 
values of education. As noted above, we call this kind of student "the scholar." And as mentioned previously, this 
ideal student is rare, indeed. 



Consider now the type of student who is primarily attracted to a particular practice or profession (say, healing people 
or designing high quality items in the cases of medicine or engineering, respectively), but who sees no (or little) 
intrinsic value in knowledge or understanding per se. This individual will require that all the knowledge and 
understanding worth acquiring must be relevant to his or her professional goals. In dealing with teachers and texts, 
this kind of student, who we have called "the professional," will be motivated to learn, but only on the condition that 
the teaching (and the grades) is clearly connected to becoming a certain kind of professional. 

It has often been said that the best motivation for learning something is intrinsic motivation (see, e.g., Jerome Bruner, 
1960). In the strong sense, however, this is false. In caring strongly about achieving some good internal to a practice 
(say, healing people), then a rational agent would care strongly about acquiring the means to achieve the goal. This is 
roughly the principle (C3) of practical rationality. One of the best motivations for learning organic chemistry is the 
belief that it will be really useful in the medical profession. The professionally-oriented student need not intrinsically 
value organic chemistry, but her respect for its utility will lead her to learn it very well indeed. 

It should now be clear that differences between the scholar and the professional account for much of the differences in 
what is learned and how it is learned. Because their primary reasons differ, these two types demand and expect 
different things from teachers and the system. For example, the professional type will always be ready to demand 
from teachers how the content will be useful for her career interests. ("How will I ever use this stuff?!?") 



And these differences between types of students can take place within a single individual who strongly, intrinsically 
values the various forms of knowledge but who, to an equal degree, extrinsically values these forms. For such an 
agent, determining what to learn and how thoroughly to learn it will be a difficult trade-off (or compromise) between 
competing goods. Knowing what to expect of this bifurcated agent is a difficult task. This type of person may demand 
and expect different things at different times in ways that seem to lead to unpredictable and irregular behavior. We 
have called this kind of student the "scholar-professional." 

For each of the three types of students, who most closely fit aspects of the ideal student, we have assumed that their 
motivation for economic and social status has little influence on their interaction with the educational system. We 
assume that their primary and dominant motives to be either the intrinsic and/or extrinsic valuing of educational 
benefits. Recall, however, that the surrogate educational benefits (grades, test scores, diplomas, etc.) function to 
distribute non-educational social and economic benefits. Now it is clearly possible and probable in our individualistic, 
wealth- and status-oriented society (though clearly not necessary) for a rational agent to be primarily motivated to 
acquire the various social and economic goods that help make life more enjoyable (to an extent). As Toqueville noted. 



it is also possible and probable that a rational agent may come to see that acquiring a differentially greater share of 
surrogate educational benefits is a comparatively reasonable means to acquiring a differentially greater share of social 
and economic benefits. (It is a highly risky, if not downright irrational, strategy to count on just being lucky.) 

Suppose, now, that this kind of rational agent neither extrinsically nor intrinsically values pure educational benefits 
(knowledge, skills, understanding, etc.). This student, who we have called "the status and wealth seeker" (one kind of 
"systemic" student), wants the degree (or the grade) only because it is a reasonable means to social and economic 
benefits. But this kind of student will regard the acquisition of knowledge and understanding as an arbitrary hurdle or 
obstacle to getting the diploma (and then the goods). Though the status and wealth seeker will want the grade and 
eventually the diploma, he or she will regard the learning as utter drudgery — something to be done as minimally as 
possible and something to be forgotten as soon as it is practical (i.e., as soon as the grade is assigned, the degree 
received, or the SAT test taken (Note 15)). This type of "student" will be like a chameleon to his or her teachers. For 
this type's public display will be like the scholar but with a fair emphasis on "brown nosing" behavior. Privately, 
however, he or she finds it all a rather disgusting game to have to play, though one played with typical thespian 
resources. When it is possible to avoid detection, the status and wealth seeker will lie and cheat, plagiarize, steal, or 
buy the necessary work (term paper mills), and do anything that will prevent other students from receiving higher 
grades than his or her own. 



Though morally rather unattractive, we should expect little else from a rational agent who sees the institutional norms 
of schooling and the social tradition of academic education as basically arbitrary matters. As we shall see, the 
normative principle connecting the educational and social and economic systems strongly encourage status and wealth 
seekers to remain in the educational system when their talents and capacities might be more productively — not to say 
morally — engaged in pursuits outside it. But the development of this kind of person is an unintended effect of our 
adoption of an educationally-based meritocracy. 



There is yet a final kind of systemic student to identify: to save space (since they are somewhat different, but 
motivationally the same in school), we called this type "the indifferent/hostile student." This is the "student" who 
neither intrinsically or extrinsically values pure educational benefits nor expects (or wants) a relatively greater share 
of social and economic benefits brought about through the pursuit of higher grades or diplomas. These individuals 
may very well want an abundance of material wealth and social respect; but they either disdain the effort (and 
charade) the status and wealth seeker employs in securing surrogate educational benefits or else views the institution 
of schooling with repugnance. 



How, then, could it be rational for the indifferent/hostile student to remain within the educational system and not drop 
out? The answer easily could be friends and expectations. The indifferent/hostile student's friends are in school and he 
or she has a primary motivation to be with them. Alternatively, such persons know that the family and society at large 
expect them to be in school, and they strongly want to please them. Finally, it is extremely rational for a teenage drug 
dealer to want to be close to the market of other kids. All of these are plausible and, no doubt, salient reasons for 
many of our disaffected young. 

But a deeper reason for staying in school has to do with the dynamics of the educational system that dictate a 
defensive strategy for rational agents continuing in school at least through the 12th grade. As rates of high school 
completion have climbed towards 100% of the school-age population in this past century (currently about 80% of 17 
year- olds complete high school), the positive social and economic benefits associated with high school completion 
have drastically declined. Indeed, as a purely logical point, at 100 percent attainment, completion of high school in 
itself can have no disproportionate social and economic pay-off for individuals (Green et al. 1980 & 1997, Ch. 6). 
(This means that the status and wealth-seeking student is forced to go on to higher education where the pay-off 
prospects are still real.) But if completion of high school is no longer a big deal, not completing high school is an 
absolute disaster for individuals. In an educational system in which nearly everyone completes high school, being one 
of the few who drop out is a near certain recipe for a life of the lowest paying jobs with an attendant probability of 
periods of unemployment or a generally unappealing life. (How many "life-long" drug dealers manage to retire after 
leading a "work" life free of misfortune?) The indifferent/hostile student, if rational, is compelled to remain in school 
out of defensive necessity (see also Thurow, 1975). 

But since the indifferent/hostile student is indifferent or hostile to both the acquisition of knowledge and 
understanding and to the relatively greater social and economic pay-off of advanced formal education, this systemic 
type of student will have little reason to come truly to grips with the curriculum or even to engage in the status and 
wealth-seeker’s charade with teachers. They have reason only to avoid the disaster that confronts the school dropout. 
And once they are compelled to be in school where they, at bottom, do not want to be, their expression of frustration, 
boredom, and hostility is quite understandable, if not potentially explosive (witness the rash of school shootings, etc.) 
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The five general kinds of students can, thusly, be categorized in terms of the kind of motive and the (comparative) 
strength of the motive (see Table 1 below). What it shows is that rational students can have a variety of different kinds 
of reasons for dealing with the educational system. For each type of student, the reasons they have will provide the 
rational justification for their action strategies. (And here we must stress the equally important fact that many students 
interact irrationally or non-rationally with the educational system, but still they mimic in large part the behavior of the 
status and wealth-seeker and the indifferent/hostile students: the two kinds of systemic students. They just are not as 
consistent and clear-minded about why they act as they do.) Moreover, a typology of the kind that we offer here calls 



for more and better empirical research to determine more precisely the relative proportions of each kind of rational 
student at the various levels of the educational system and their less rational counterparts. (Note 16) Such research 
will improve our understanding of what kinds of reforms are likely to be effective in working with each kind of 
student and in what degree. As we turn to the final part of this paper, we shall address our comments to other 
important, more philosophical, aspects of this issue. 



Table 1 

Certain Types of Student With Respect to Kind and Strength of Motive 





Kind of Motives (Reasons) 




Intrinsically Motivated by 
Knowledge and 
Understanding 


Extrinsically Motivated by 
the Utility of Knowledge 


Extrinsically Motivated by the 
Social and Economic Benefits of 
Educational Attainment 


Scholar 


Dominant-Strongest 


Weak 


Weak 


Professional 


Weak 


Dominant-Strongest 


Weak 


Scholar- 

Professional 


One of the Two 
Dominant/Strong 


The Other of the Two 
Dominant/Strong 


Weak 


Status and Wealth 
Seeker Strongest 


Weak 


Weak 


Dominant/ 

Strong 


Indifferent/ 

Hostile 


Weak 


Weak 


Weak or Negative Dominant 
Strongest 



IV 

Student Rationality, Educational Excellence, and 
The Educational Reform Movement 

Through the framework we have established, we hope we have made a start in establishing that rational student action 
can have a powerful effect on the eventual results of the educational reform movement. Yet it still seems rather odd 
that there is very little mention in the educational reform literature on potential student reaction to reform efforts. And 
what little there is comes mainly from within the educational profession itself (for example, the so-called "middle- 
school philosophy"). But perhaps this oversight should not be overly surprising after all. For several concerns and 
assumptions have been at work in the educational reform literature from its inception with "The Nation at Risk" 
report. These are concerns and assumptions that have dominated and directed the ensuing discussion and debate by 
policy makers over what reforms to pursue. 



The first of these concerns has been the economic competitiveness of the various states, and the nation as a whole, as 
the post-World War II dominance of the United States underwent successive challenges on the world market, 
especially in the 1970 — 1990 period. It was during this time that the educational reform movement first emerged and 
gathered steam to the point that the 2000 state and national elections made educational achievement the paramount 
political issue facing the country. (Now, of course, education has been overshadowed by the events of 1 1 September 
2001 .) In other words, the policy makers have shackled the cause of educational reform to the fortunes of our 
aggregate economic activity. But this view of the purpose of education — to supply a schooled workforce to meet the 
needs of an increasing technological world — almost guarantees that the young and their purposes for undertaking 
schooling will be lost from view. Yet almost no one, we would conjecture, goes to school and strives (or fails to 
strive) for higher levels of educational achievement because it is good for the American economy. Because of this 
tunnel vision driving educational reform, it is difficult to find a policy maker on whatever level who has asked a most 
basic question: What is in it for the student? 



Second, in viewing high levels of educational achievement as the principal means to attain economic salvation, the 
educational reform movement tends to assume that the young are monolithic in nature. Indeed, the ongoing concern 
with school drop outs has one primary aim: get them back in school (or keep them from dropping out) so that they can 
be "regular" students once more. But, as we have argued, there is no such thing as a "regular" student. There are a 
variety of student types who vary in their goals and strength of motivation, and by anyone's estimate most of these do 
not resemble the three types of an ideal students. Enticing drop outs back into schools will do little more than 
reinforce this monolithic view of students and perhaps lower achievement levels further. Early school leavers, many 
of whom have subjectively reasonable beliefs in light of their limited experience, can potentially be shown that it is 
definitely in their long-term social and economic self-interest to return to (or stay in) school, if only to avoid the 
disaster that is their eventual destination. But such policies and programs to encourage them to return to school will 
only swell the ranks of indifferent/hostile students (and to some extent status and wealth-seekers), which is why most 
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of them left school to begin with. It is not that we should be indifferent to the plight of the drop out, for it is serious 
and real. Rather, we need to understand that keeping those young in school who have no taste, at least at the moment, 
for academic work is hardly a recipe for higher levels of achievement in the aggregate and for harmonious school 
environments. 

Again, for any proposed educational reform, we must ask one of the most important policy questions: What is in it for 
the student? And this is not simply a crass, egoistic type of question to raise either. Some ends for which humans act 
are ultimate ends concerning " internal" goods (about which more below) and can be shown to be worthy of both 
rational and moral choice. Thus, the scholar, the professional, and the scholar/professional type of student who, in 
pursuing the ideals of knowledge and understanding and/or service to fellow citizens and humanity, are engaged in 
pursuits and practices that are rationally and morally laudatory. But still, it makes sense to ask from their point of view 
what is in any proposed educational reform for theml Yet these kinds of students are probably in the minority of 
students in the educational system. The educational system, because of the normative principle linking the educational 
and social and economic systems, is replete with systemic status and wealth seekers and indifferent/hostile students. 
(And it is well to notice that many so-called professionally-oriented students are actually status and wealth seekers. 
How many students, for example, would pursue the medical or legal professions were they of low pay and low 
prestige?) Thus, the success of implemented and proposed educational reforms rests largely on their ability to engage 
the interest of the status and wealth seeker and the indifferent/hostile student. 

But now consider the nature of the proposed and implemented reforms. For example, there are reforms that lengthen 
the school day and the school year. There are reforms that call for a more demanding school curriculum and higher 
graduation requirements. There are reforms that call for the ending of the "social promotion" of students from one 
grade to the next higher, regardless of school performance. There are educational reforms that mandate a minimum 
grade point average for participation in extracurricular activities such as sports. And there are state standard s-based 
exams that govern high school graduation. Now few of these reforms are likely to affect adversely the interests of the 
scholar, the scholar/professional, and the professional. Of course,the professional and scholar/professional may resent 
more and higher requirements in areas irrelevant to their interests. However, one of these reforms — standard s-based 
examinations — could well impact adversely the interests of the scholar and the scholar/professional. That would 
especially be true if the testing, as likely, drives classroom instruction in a teacher and school "accountability-based 
system" now being implemented in state after state. For as teachers and administrators feel accountability pressures to 
raise average test scores, they are likely to target instruction at the "least able" students (who have the most room to 
improve). We should expect an attendant result to be the lowering of the level and content of teaching. In this way, 
state standards-based, assessment and accountability systems could help turn many ideal students into cynical, 
resentful ones or else help drive them into the private sector of the educational system. Alternatively, even the scholar 
may become frustrated by a different kind of reform found in the Ontario, Canada schools that expects students to 
master a far more demanding curriculum in much less time — a form of "curricular intensification." 

But, now, what is in these reforms for the status and wealth seeker and the indifferent/hostile student? The answer is 
little else but pain and suffering. The most rational response of the status and wealth seeker is to seek shortcuts and 
end-runs around these reforms when necessary and to try to frustrate their intent whenever possible. For the 
indifferent/hostile student, there is but one rational strategy: sabotage at all times by refusing to play the game when 
there is nothing at stake personally. And when high school completion is at stake in those school districts and states 
with test-passing requirements, the indifferent/hostile student, if rational, will put forward the minimal effort 
necessary in order to pass the test, thus only grudgingly avoiding the plight of the drop out. 

Of course, these are but a few ways to frustrate and undermine attempts to reform the educational system. Many of 
our more rational students, in the main, may not know and understand in any great depth the various ways we have 
come to structure our experience of the world, but they can be very clever and resourceful in ensuring that they never 
come to that truly educated state. And can we blame them? Perhaps in a world that emphasizes the intrinsic value of 
pure educational benefits and their service in upholding the ideals of democratic citizenship, we might well place 
blame on the more rational students who foolishly waste important educational opportunities. But can we place blame 
on such students in a world that emphasizes formal schooling as the prime means to economic dominance as a nation 
and "making it" socially and economically in personal terms? (Can we even place blame on such students in a world 
that uses grades "earned" to sort students at each level? Even the scholar and professional types will see how getting 
high grades is strongly related to getting into higher levels of the system.) (Note 17) We think not. For it is not our 
students who have placed great emphasis on the purely instrumental value of formal education for both the economy 
and the social and economic standing of the individual. Rather, it is the truncated vision of the erstwhile educational 
reformers and of a society that apparently cares more about the credentialed symbols of educational achievement than 
about the intrinsic and extrinsic value of pure educational benefits in leading a good and worthy life. We have, in 
other words, unerringly established an educational system and a set of social and economic incentives that are 
guaranteed to deliver marginal educational achievement and to create resistance to any real and meaningful 
educational reform. We should not be surprised, therefore, to find so many status and wealth seekers and 
indifferent/hostile students in our schools. For in creating an educationally-based meritocracy, we have done 
everything we could to encourage their development. 

Education in this way, rather than seriously pursued, becomes a rather cynical game to be played. The problem is not 
that we fail to value education. Clearly, Americans (and Canadians) do. The problem, instead, is the way we value 
education. 




This points to the deep and final incoherence that lies at the heart of the educational reform movement. By shackling 
the drive for educational excellence in education to the cause of competitiveness in the world market, we are likely to 
achieve neither. For at the heart of the reform movement, there exists a defective understanding of the nature of 
educational goods, true educational standards, and how excellence is to be promoted and sustained. The cause of this 
defective understanding is an intellectually-derived, moral tradition that runs deep and powerful in American life: 
"moral individualism." Moral individualism is, as Stoutland notes, "...a comprehensive, individualistic moral theory 
about how individuals should live their lives and relate to society" (1990, p. 107). At its best, it can provide a rationale 
and basis for understanding society as a cooperative endeavor in which some self-sacrifice is required of all for the 
benefit of all. At its worst, and this is its modal tendency, moral individualism promotes egoism and the satisfaction of 
individual preferences even at the expense of others. Since, as we claim, moral individualism is clearly reflected in the 
thinking of the educational reformers, it will profit us to examine some of the beliefs that comprise it. 

Moral Individualism and the Educational Reform Movement 

Again following Stoutland, "...moral individualism understands the good as anything that satisfies an individual's 
desires, interests, or preferences. This implies that all goods must be individual goods, that is, goods for, and 
assignable to, particular individuals, since all desires, interests, or preferences belong to particular individuals" (1990, 
p. 119). It follows from this that a social good can only be one that satisfies the preferences of most of society's 
members because a social or public good is only the sum of individual preferences. The common good or public 
interest is only, on this view, the sum of individual preferences. The common good or public interest is thus 
necessarily reducible to the private interests of individuals. The provision of education to all, then, is a social or public 
good if it can be shown to be in the public interest, that is, the interests of most of society's members. Although 
altruism — or the self-sacrifice of one's own private interests for the private interests of others — can find a place within 
moral individualism, that place is necessarily precarious. For happiness or the good life in moral individualism is a 
life in which one's own private interests are maximally satisfied. Thus, appeals to the common good must be couched 
in terms of appeals to the private interests of the many. And when individuals do not see a particular candidate for a 
public good deserving of their own support (i.e., in their own private interest), the majority who do may be forced to 
compel self-sacrifice on the part of those who do not (e.g., compel them to pay taxes for the support of formal 
education). As, Stoutland succinctly puts it, "What is distinctive about moral individualism is not that it assigns no 
important role to society, but that it regards society, as wholly instrumental to goods for the particular individuals who 
are its members.. .Societies [on this view] do not constitute preferences; their role is to satisfy them" (1990, pp. 1 20). 

We are now in a position to appreciate the way in which the educational reform movement is ensnared in the trap of 
moral individualism. In order to appeal for higher levels of achievement in — and more money and accountability 
for — education, the reformers have been forced to explain how higher achievement is instrumental to the satisfaction 
of the sum total of private interests. This they have done through the appeal to the economic competitiveness of the 
nation and the related theme of a forewarned unilateral " educational disarmament." (Note 18)> 

But while the reformers carried on the battle-cry for educational reform to society at large, they left their rear 
unguarded on the issue that we have addressed: the interests of students. But should they finally address the various 
rational interests of students, then what, within the confines of the value framework of moral individualism, can they 
say? First of all, no appeal is possible to education as an intrinsically valuable good, since in moral individualism 
nothing has intrinsic worth. Here something is good only if it satisfies some private preference. Second, for them no 
appeal is possible to education as an extrinsically valuable good, a good which by definition leads to an intrinsically 
valuable good, since again in moral individualism nothinghas intrinsic worth. For such a view, something is good 
only if it satisfies private (short term or long term) preferences. And if a student has no preference for higher 
educational achievement (short term or long term), no instrumental appeal is possible. Third, because the reformers 
have tied education to the purpose of economic competitiveness — rather than to the non-individualistic ideals of full- 
blown rationality and democratic citizenship — no appeal to student responsibility is possible. And fourth, appeals to 
students to achieve highly for the public good (economic competitiveness) are definitely likely to fall on deaf ears. 
Any tendencies toward self-sacrifice in our moral individualistic society are fairly diminished by the teenage years. 
(The patterns of altruism that clearly do remain are no doubt testaments to the staying power of those more 
communitarian social institutions such as the family and church.) And finally, unlike the paying of taxes, it is doubtful 
that we would compel the young to self-sacrifice by threatening fines and prison sentences for low achievement — 
though in a few places state legislators have actually introduced bills to deny a driver's license to errant students! 

Thus, the only appeal to students that can work is to their self-interest. The educational reform movement has no other 
real recourse. But in the appeal to narrow self-interest we have, given the structure and dynamics of an educationally- 
based meritocracy supported (by and large) by a bedrock of moral individualism, the very instrument that delivers the 
educational mediocrity the reformers decry. What the reformers fail to see is that the structure and dynamics of the 
educational system are actively forming and encouraging student preferences that run counter to the creation and 
sustenance of educational excellence. One might say that the system is creating status and wealth seekers and 
indifferent/hostile students. Excellence in education has nothing at all to do with the external goals of sustaining a 
competitive economy or materially enriching individuals. But that is something very difficult for those of us caught 
up in our individualistic culture to understand. And it is certainly the central defect of the educational reform 
movement. It leads to the incoherence of which we spoke. Indeed, excellence in education is not something that can 
fit into the framework of moral individualism at all. 
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Educational Excellence and the Idea of a Public Good 



Central to this understanding is the act of recognizing that there are some kinds of goods that are irreducibly public 
goods — goods that cannot be privately assigned to or appropriated by individuals alone. Alasdair MacIntyre in After 
K/r/we (1981) establishes this in his account of social practices and his distinction between "internal” and "external” 
goods. For Maclnt 30 ‘e, a practice is "...any coherent and complex form of socially established cooperative human 
activity through which goods internal to that form of activity are realized in the course of trying to achieve those 
standards of excellence which are appropriate to, and partly definitive of, that form of activity" (1981, p. 175). 
Examples are complex games, true professions, and, not least, the various arts and sciences. What is common to all of 
these is the fact that they each contain standards of excellence (norms) that define their corresponding activities and 
what it means to be a skillful participant in them. Moreover, they each require no mean effort to master (and typically 
require forms of apprenticeship); and in many cases they cannot fully be appreciated (or judged) except by those on 
the inside of the practice. On the other hand, goods are external to a practice if they can be secured in some other 
fashion than through the practice itself. Status and wealth are obvious examples. Since external goods can be 
appropriated by and assigned to individuals, they are "...characteristically objects of competition in which there must 
be losers as well as winners" (MacIntyre, 1981, p. 178). And although internal goods are "...the outcome of 
competition to excel . . .it is characteristic of them that their achievement is a good for the whole community who 
participate in the practice" (MacIntyre, 1981, p. 178). In other words, they are not reducible to the good of particular 
individuals. 



The achievement of knowledge is one such example of an irreducible public good that benefits the community. And 
note that in the transmission of knowledge through education, teachers do not give up something when students come 
to understand. The economics of exchange relations do not apply here (nor, as Green et al., 1980 and 1997, note, is 
knowledge subject to the economist's notion of "decreasing marginal utility"). 



Now Maclntj/Te's account of practices allows us to establish exactly why the educational reform movement has a 
flawed and defective vision of educational excellence. In understanding the practice of education as merely 
instrumental to satisfying certain desires (i.e., economic competitiveness or individual social and economic well- 
being), they reduce educational benefits to external goods. But excellence in education can only be understood by 
reference to the public standards internal to the practice. (Note 19) In this way, pure educational benefits are not 
goods because they satisfy individual preferences. They are goods because they are specified by the standards of 
excellence internal to the various forms of knowledge that we have achieved over time. They can be realized only by 
engaging (to a considerable degree) in the practice of education in its own terms, by coming to see its point in and of 
itself, and therefore only by submitting oneself to its discipline. (Note. 20) 



But in thinking of educational benefits as merely the means to satisfy aggregate or individual preferences, the 
educational reform movement rules out the possibility of understanding education in its own light. In so doing, the 
policy makers necessarily fail to capture the very nature of educational excellence from the outset. 

The Practice of Education and Educational Reform 

In taking the practice of education in its own light seriously, we quickly begin to reconsider the meaning of questions 
such as "What is in it for the student?" That question, we should note, arose primarily within the framework of moral 
individualism. Instead, we need to think about educational reform in terms of the institutions whose role it is to 
susmin the virtues inherent in the practice of education. Again, if we look at the educational system today, we witness 
an institution that undermines, rather than sustains, the pursuit of educational excellence. Individualist thinking simply 
lacks the capacity to understand how important the structures of society are in forming individual preferences in the 
first place. The normative principle that governs the link between the educational system and the social and economic 
system simply instantiates the norm that the road to riches is through high grades, rather than emphasizing the 
intrinsic and extrinsic value of educational benefits. Instead of taking the interests of the status and wealth seeker and 
the indifferent/hostile student as givens and then asking how we can transform their preferences into those of ideal 
students, we should be asking a far different question (unless we want to continue to blame educators in the interest of 
political mileage). For the fact remains that once formed, preferences are difficult to change. Rather, we should be 
asking; "What is it about the educational system that leads to the development of status and wealth seekers and 
indifferent/hostile students?" In other words, transforming the educational system may be the only way to transform 
individuals. Only then does the direction of real and meaningful educational reform — and the true meaning of 
educational excellence — become evident. 



This is the direction that has been urged by Green et al. (1980 & 1997, pp. 164 -168). It can be re-framed more 
directly here. Stated simply, we must weaken (if not abandon) the normative principle that differentially rewards 
educational attainment (grades, degrees, and diplomas, etc.). In other words, if there were no longer any major pay off 
economically and socially for educational attainment via the educational system per se, then education through the 
educational system could be unwaveringly pursued for the intrinsic and extrinsic value of pure educational benefits. 
Moreover, if educational attainment in and of itself was less decisive for life chances, then the social and economic 
compulsion to complete, for example, high school would disappear. Simply consider what this would mean for the 
indifferent/hostile student as well as, remarkably, for the drop out. By weakening (or even severing) the connection 
between the distribution of surrogate educational benefits and the distribution of social and economic benefits, the 
motivation to pursue educational attainment for purely defensive purposes (to avoid the current plight of the drop out) 



is gone. The indifferent/hostile student would be given a real choice concerning the future. And many of them would 
exercise such a choice, at least for the time being, by dropping out of the educational system. With the absence of 
unwilling and resentful students, we should expect plummeting rates of school violence and the restoration of a 
healthy climate for learning. 

But with a higher percentage of school leavers in the secondary school age population, the social and economic costs 
of dropping out are greatly reduced. Dropping out is a personal disaster and social stigma when only very few drop 
out. In a world of many drop outs, employers cannot routinely screen for formal, but often suspect, surrogate 
educational credentials. 

There is an equal implication for the status and wealth seekers. If the pursuit of educational credentials were no longer 
the primary route for "making it" in life, their reasons for remaining in school and engaging in manipulative and 
deceptive behavior patterns simply collapse. If "making it" is truly their goal, then they would be free to expend their 
doubtless ingenuity outside the schools in other, hopefully more worthy, pursuits in striving to reach it. 

But with the educational system reduced, thusly, in size — ^retaining and, most importantly, easily re-admitting only 
those who wish to profit from the disciplined pursuit of knowledge, understanding, and callings — our society would 
be forced to understand that the demands of real and lasting educational reform greatly exceed the current, ephemeral 
attempts to tinker crudely and blindly with the educational system as it exists. If we continue our present course, we 
may realize fleeting gains of a few points on test scores. (Note 21 ) Indeed, we may even be compelled to face the fact 
that there might be a multitude of pathways to make the transition from youth to productive and competent adulthood, 
only one of these through the educational system as such. (Note 22) Quite apart from the now fashionable so-called 
"middle school philosophy" developed to deal with disaffected students, we may need to think about returning to the 
concept of the retired grammar school to provision all students with elementary/intermediate literacy, civic, scientific, 
and mathematical knowledge and skills, along with basic computer skills. Upon completion, students might then 
choose between continuing in the academic educational system or opt out for more practical educational 
opportunities. Except for the few advanced jobs and career categories — in relation to the entire population — it is 
questionable that we need hordes of high-tech people to run the economy. (Few of the status and wealth seekers and 
indifferent/hostile students are currently destined for these positions anyhow. Indeed, more of them might actually end 
up in these positions by easily re-admitting them into the educational system when they are more ready.) 

But the need to open new pathways (and in some cases re-open old ways) in the transition to adulthood would raise a 
host of issues about how to structure strong educational and economic policies to foster different kinds of practical 
learning experiences for youth directly in the workplace and other social settings. For it has never been the case that 
the young have little interest in learning per se. Curiosity and the thirst for learning are universally natural to the 
young. Rather, the problem is that we have compelled young people to pursue one kind of learning - 
scholastic/academic learning — for ever increasing amounts of time without regard for its perceived relevance to them. 
The clear and convincing result is now an educational system awash with status and wealth seekers and 
indifferent/hostile students, not to mention their less rational counterparts. By creating multiple pathways to adulthood 
that feature practical, hands-on, experiential learning within a "real world" context, we can develop arenas that will do 
much to foster moral attachment, real learning, and a "conscience of craft" (Note 23) (for those currently disaffected 
with academic culture and practice). Rather than stigmatizing such academic "drop outs," a multiple pathway 
approach might be far more appealing to a majority of adolescents of all social classes (especially since it would not 
foreclose the option of dropping back into the academic educational system later on). (Note 24) It is true that many 
might not ever return to academics, as such. However, they most likely would end up with far more marketable skills 
than our current crop of disaffected high school graduates. With such skills, the nation's economic competitiveness 
might be heightened beyond the educational reformers' dreams. (There is more than a touch of irony in this.) 



Such extra- systemic educational reform would not be easy by any means. (Note 25) Indeed, it would require the 
creation of a public consciousness that education - in all of its forms and throughout a lifetime — must be seen to be a 
society-wide responsibility, not just the currently, and mainly age-segregated institution of schooling. The educational 
system, if allowed, can easily succeed in the pursuit of true educational excellence if its mission is appropriately 
construed as the producer, guardian and transmitter of fundamental cultural and scientific understanding. It cannot, 
however, be all things to all people as we now pressure it to be. For far from achieving continuing competitiveness in 
the world ~ so far an event that has as much to do with luck, fair-enough economic design, strategic collapse in the 
old Soviet Union, and immigration -- the current spate of educational reforms will do little to move us truly ahead on 
the road to educational excellence. And that is the ultimate lesson to be learned from raising the question of the 
student in educational reform. 

Notes 

^ The inelegant, but descriptively accurate, term "studenting" was originally introduced by Gary D Fenstermacher 
(1986) in "Philosophy of Research on Teaching. " It refers to those activities of the student often necessary for student 
achievement. 



ERIC 



“ We believe that the following sketch of an "ideal" student is widely held, but our subsequent arguments do not rely 
on its being widespread. 
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- This view paraphrases that of Paul H. Hirst in his justly famous essay, "Liberal Education and the Nature of 
Knowledge" (1974). Howard Gardner's The Disciplined Mind (1999) gives a related formulation. 

4 Hirst, Ibid. 

^ Thomas F. Green (1999) in Voices: The Educational Formation of Conscience explicates the major difference 
between what he calls strong and weak normation. In this instance, ideal students become strongly normed to the 
standards of excellence in education (and to the academic purpose of schools in so far as schools support these 
standards). Indifferent students, as we shall see, are neither strongly normed to the standards of excellence in 
education nor to schools, though they may be compliant with school rules and routines (weak normation). Hostile 
students are neither strongly nor weakly normed to the standards of educational excellence or the schools. Indeed, 
they tend to be defiant of both. 

^ Steinberg, et al. in Beyond the Classroom: Why School Reform Has Failed and What Parents Need to Do (1996) 
studied 20,000 teenagers and their families in nine different communities over a ten year period. From a 
psychologist's point of view, he examined the "engagement" of young people in schools (with "engagement" defined 
as "the degree to which students are psychologically "connected" to what is going on in their classes" (p. 15).) In his 
sample, he discovered that around 40% of the teenagers from all social classes were "disengaged" (p. 67). viliile there 
are strong reasons to prefer Green's (op. cit.) notion of strong and weak normation in identifying types of students and 
their motivations, Steinberg's "disengaged student" may serve as a proxy to our indifferent/hostile types of students. 
We also take strong exception to Steinberg's recommendations for "re-engaging" students. Steinberg's psychological 
framework also fails to mark out how students who are disengaged might be acting quite rationally (from their point 
of view) in the educational system. Accordingly, he omits any mention of a connection among rationality, leading a 
good life, and avoiding harm. 

^ See Ericson & Ellett (1990) "Taking Student Responsibility Seriously." 

^ Ibid. 

^ Postman (1995) in The End of Education strongly takes to task the current economic rationale for public education 
and calls for a new, far more noble, "metaphysics" of education. 

This section is strongly grounded in the work of Green et al. (1980 & 1997), Predicting the Behavior of the 
Educational System, especially Ch. 6. 

See Martin Trow's (1961) classic, "The Second Transformation of American Secondary Education.". 

This use of the term "surrogate" acknowledges that grades, for example, are neither interval-ratio scales nor even 
ordinal scales. Their "sloppiness" as scales is what makes them an efficient medium of exchange. 

1^ The United States, we should note, is far from a perfect educationally-based meritocracy. Typical of capitalist 
economies, the U.S. legitimates social and economic inequalities based on personal luck, perseverance, and even such 
things as a winning personality. Increasingly, however, formal educational attainment governs entrance into career 
and earnings networks and hierarchies. 

The supporting arguments for regarding the normative principle as a principle that legitimates subsequent 
inequalities in social and economic goods among persons is given in detail in Green et al. (1980 & 1997), pp. 42-45. 
The point to grasp is that the relation between (surrogate) educational benefits and non-educational social and 
ecconomic benefits is not merely a strong, positive (causal) one, but also is a justified or authorized one in our society. 

The popularity of Scholastic Assessment Test coaching firms, such as the Princeton Review and Kaplan, is a 
testament to this widespread attitude. They typically guarantee higher SAT scores through emphasis on test taking 
skills and strategies, not the acquisition of knowledge and understanding. 

Again, see Steinberg (1996), op. cit., whose survey evidence comes closest to providing this. 

See, for example, Howard S. Becker (1989), "A School is a Lousy Place to Learn Anything." 

This phrase headlined The Nation at Rw/r report (Gardner, et al., 1983). The rhetoric places education — and its 
support — on a par with national defense in terms of national importance. As Green et al. (op. cit.,.,pp. 147 - 156 ) 
point out, this is the strongest possible appeal for the support of the educational system. 



Such standards of excellence internal to education are not to be confused with the generally woeful, rarely even 
rationalized and certainly not justified, state educational standards of the educational reform movement. 

As opposed to the individualists identification of the good life with the satisfaction of private preferences, more 
communitarian-type views from Aristotle to MacIntyre and Green locate the good life in the realization of irreducibly 
public goods embedded within irreducibly social practices. For such views happiness is the zestful exercising of basic 
human capacities (e.g., intelligence or inventiveness) in the pursuit or cultivation of the kinds of excellences 
appropriate to a practice of a given kind. Though the excellences of building a common world, building a family life, 
or of education require different kinds of activities or performances, the performance ’’commanded," as it were, in 
each is a kind of virtuosity. Thus, the goodness or badness of the performance is a matter of objective evaluation in 
relation to the standards of excellence that are definitive of the communally sustained practice. 



This is not to reject the place and appropriateness of external goods such as wealth or status in a full life. For these 
may re.sult from a life devoted to the pursuit of excellence. Rather, it is simply to note that external goods cannot be 
the aim of the good life on communitarian grounds. 

And again, we should stress, while individualism understands society as just instrumental to the satisfaction of 
individual preferences, communitarians see society as essential. For even while in competition to excel, 
communitarians understand that internalization of the norms of the practice by all who compete is necessary to sustain 
its flourishing existence. (This is why truth, courage, and honesty are central virtues to most practices. Dishonesty and 
bad faith may often be the quickest route to fame and fortune, but they undermine the deep layer of social solidarity 
that forms the foundation of any practice.) 

The recent Rand study (Grissmer et al., 2000) suggests that heavy-handed accountability measures can squeeze out 
some gains, after all. But even here, the data reporting of some states, such as Texas, is suspect. See, for example, 
Linda Darling-Hammond (1999, p. 3). 

The Coleman Report on youth in transition, though now long-forgotten, remains the most serious and thought- 
provoking study in alternatives to the current regimen of growing up in America. It needs to be re-visited. See James 
S. Coleman (1974). 

See Green (1999), op. cit. Such settings, in other words, would be able to foster strong normation and other 
attachments necessary for leading a good and productive life. Weak normation (at best), anomie, and defiance are 
currently the phenomenological states of the indifferent/hostile students in our schools. 

In some ways our view is compatible with the views of Howard Gardner, "Getting There," The Disciplined Mind, 
pp. 214- 240. We and Gardner both advocate the end of the monolithic educational system. For even schools, as such, 
might have multiple pathways as is common in Scandinavia, Germany, other parts of Europe, and Japan. (Note that 
many of these societies are far more egalitarian than our own.) But Gardner only sees the high technology pathway of 
Bill Gates, Louis Gerstner, and others. Gardner fails to consider the possibility and viability of non-school pathways 
that would enable students to enter the career market in a variety of ways. But even if restricted to schools, a multiple 
pathway approach would have distinct advantages. For the differing "exit grades" of each would inhibit the continued 
existence of a common coinage (medium of exchange) as represented by the current formal domination of the 
Carnegie Unit system and informal, but real, pressure exerted by colleges and universities to specify the intermediate 
and high school curriculum in a manner to meet their own needs. 

For example, it may even require an examination and restructuring of the reward schedules for certain learned 
professions such as medicine and law. Like the academic profession (some of whose sub-specialties deserve similar 
attention), they already command considerable social respect. But when coupled with relatively high economic pay- 
off, their probability of attracting the strong attention of the status and wealth seeker escalates enormously. The 
proliferation of "professional ethics" courses in professional schools is a testament to the fact that the high professions 
are in imperiled in this way. For the aim of status and wealth seekers is to profit them first and foremost, and only 
incidentally serve their fellow human beings. 
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Abstract 

School reform efforts aimed at promoting equity and excellence at urban school settings are 
heavily dependent upon the quality of teaching personnel that are used to deliver the instructional 
program. Social Justice and other public policy issues related to equity and excellence at urban 
schools have begun to examine the impact that teacher absenteeism, and by extension the 
reliance on substitute teachers to deliver instructional might have on educational attainment. 

This study combines school district data gathering mechanisms on teacher absence rates at school 
sites with Geographical Information Systems (G.I.S.) to map the association between a school's 
geographical environmental space and the propensity for teacher absence. The disparity between 
teaching resources as delivered by the school district vs. teacher resources as actually received by 
students in the classroom via teacher absenteeism is examined in the context of schools located in 
positive (high income) and negative (low income) geographical space. The study concludes that 
there is a strong association between the geographical quality of the school site setting, teacher 
absenteeism, and the reliance on substitute teachers to deliver instructional programs. Disparity 
in teacher absenteeism rates across large urban geographical areas threatens the promotion of 
equity and excellence in the schools by attenuating or lessening the effect of school resources to 
support instruction and amplifying the risk factors of students in the classroom. 



Introduction 

Assemble all the worst things in America- gambling, liquor, cigarettes, and toxic fumes, sewage, 
waste disposal, prostitution- put it all together. Then you dump it on black people and their schools. 

(Kozol, Savage Inequalities y p. 17) 

The above insight is from a book that attempts to document the bleak environmental or geographical space context of 
some of our urban school settings and its possible impact on the community- especially children and adolescents that 
live in the community. Schools located in what might be termed. Negative Geographical Space or N.G.S. 
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(Bruno, 2000) are particularly troublesome with regard to promoting equity and excellence in their instructional 
programs. One educational research finding that seems to remain constant is that school reform efforts aimed at 
promoting equity and excellence in the schools are strongly dependent upon the quality of teaching personnel needed 
to deliver the instructional program to students in the classroom. 

It is common knowledge that urban classroom teachers also tend to work under more stressful working conditions 
therefore use their sick days more frequently. In general, urban classroom teachers, teach more students a day, teach 
less academically prepared students, teach in more dangerous, high crime geographical areas of a large urban area, 
and generally lack teaching and instructional materials to deliver a quality instructional program. In essence, the 
geographical or environmental context of these school working conditions, even more that student poverty, are 
strongly associated with teacher attendance at the school site, their morale as professional educators, and a their sense 
of teaching efficacy in the classroom (Corcoran, Walker, & White, 1988). 



With its [the urban school’s] outdated textbooks and crumbling, dirty facilities, it operated in 
conditions that would not have been tolerated in other mixed or predominantly white communities. 
(Diver-Stamnes, 1995, p. 17) 



The issue of teacher absenteeism is rapidly becoming an important topic area of educational policy analysis largely 
because of its direct impact on the quality of instruction and its association with poor participation in school reform 
efforts. (Ehenberg, Rees, and Ehenberg, 1991). In an effort to explain the why to teacher absenteeism, some research 
studies have also examined the role of gender and its association with teacher absence (Scott & McClellan, 1990). All 
of these research efforts are directed towards developing a better understanding of the problem of teacher absenteeism 
and more importantly the propensity of teachers to use all of their available sick days as an "entitlement" of the 
profession. 



Because of high levels of teacher absence and the extensive use and reliance on substitute teachers to deliver 
instructional programs there is a strong attenuation or a lessening of the impact of school resources that are devoted to 
instruction. There is also a corresponding amplification of student risk factors such as poverty. These two major 
consequences of teacher absenteeism, attenuation of school resources and the amplification of student risk factors, 
make the teacher absenteeism issue one of extreme importance for urban schools. Of particular importance is the 
impact that teacher absence might have on school reform efforts aimed at promoting equity and excellence at these 
schools. From a social Justice perspective, another important issue to examine is if there is any systematic bias in the 
distribution of teacher absenteeism with higher teacher absenteeism in the low-income areas of large urban school 
districts. In essence, because educational researchers have suggested that the effectiveness of substitute teachers is far 
below that of regular teachers (Olsen, 1971). Teacher absenteeism, by extension, has implications for school reform 
and the quality of the instructional program at a school setting and policy issues dealing with improving equity and 
excellence in the schools. 

In many instances in urban school settings, a substitute teacher cannot be found and a regular classroom teacher to 
cover the classroom time period from their free or preparation period. (Locker, 1999, Mckay,1999) , Finally from a 
purely financial perspective, teacher absenteeism is extremely expensive and substitute teacher salaries have to be 
paid in addition to regular teacher salaries thus adding to the per pupil cost of education. In essence, there is an 
attenuation or lessening of school resources needed to support instructional programs at these schools- a situation that 
would not be tolerated at predominantly white, suburban schools. 



Of course there are many legitimate reasons for a teacher to be absent. Typical reasons include illness, professional 
development, personal reasons, family bereavement, etc. (Alberta Teachers Association, 1998). Sometimes as 
teachers approach retirement age and have accumulated sick days with the school district, they begin to take their 
takes off to pursue their own personal interests or outside "moonlighting" activities. Since in most urban school 
settings each teacher is permitted to "bank" 10 days of sick leave per year, the total amount of sick days for a teacher 
can rapidly accumulate over a career in the classroom. What is most troublesome for educational policy makers is 
when classroom teachers at certain school sites begin to view their absenteeism from the classroom as an entitlement 
that goes with the teaching position. The sense of entitlement has enormous implications for the cost of instruction. 
Consider the following example: 




Assume a 180 day school year: 

• 100 teachers at $75,000 (salary plus benefits) = $7,500,000 or $416 per day per teacher 

• Assume that each teacher takes 10 days of sick leave per year = 1000 teacher days 
missed or 10% of $7,500,000 = $750,000 of resources not being received and replaced 
with a substitute teacher at $200 per day = $200,000 

. The net loss to students is $7,500,000-$750,000-$200,000 = $ 6 ^ 550,000 

(thus the disparity of resources actually received by students in the classroom can be 
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quite large and significantly impact on the quality of the instructional program and 
threaten efforts aimed at promoting equity and excellence in the schools). 



In addition to the attenuation of school resources towards instruction, when this sense of entitlement is most prevalent 
or biased towards schools located in negative or low-income geographical space the very students most in need of 
instructional quantity and quality are harmed. The challenges posed to teachers and educational leaders between the 
quality of geographical space or school site location, poverty, and school issues have also been explored in 
educational research studies. (Bruno, 2000; Lippenant, Bums, McArthur, 1996). 

Purpose of Study 

The purpose of this study extends these research efforts to examine the quality of geographical space and teacher 
absence linkage since it indirectly examines how school site location impacts on access to educational opportunity. 
Specifically this study examines the association between the quality of geographical space or the environmental 
context of the school setting and the rates of teacher absenteeism at the school site for all of the high schools located 
in a large urban. 

The Substitute Teacher in Urban Settings 

Equality of educational opportunity at a school exists when a child's educational opportunity does not 
depend upon either his parent's economic circumstances or his location within the state (Wise, 1972, 
p. 146) 

As previously noted, issues related to quality of educational opportunity and the related issue of equity and 
excellence at urban schools strongly depend on the quality of classroom instructional programs. The reliance on 
substitute teachers to deliver instruction, because of teacher absence has great significance for these urban students 
because parents or the economic circumstances of these students make the school setting the only source of 
educational development. Instructional programs delivered mainly by substitute teachers, non-credential teachers, 
inexperienced teachers, etc. therefore are extremely important. Some of these issues have been previously and non- 
empirically addressed in various books on the problems of urban school setting. (Kozol, 1992). Since the quality of 
substitute teachers and not fully interchangeable with the quality of the regular classroom teacher at most school 
settings, the issue of teacher absence has both practical (in terms of cost) and theoretical (in terms of equity) 
significance for the educational policy maker. 

The regular high school classroom teacher is required to obtain a bachelor's degree in a same field that is related to his 
or her teaching assignment. This could range from the sciences, such as chemistry, to education such as special ed., to 
mathematics such as algebra II, to the fine arts such as painting. The regular classroom teacher is also required to 
complete two years of a teacher training program in a post graduate department of an accredited institution. While 
there are some exceptions, substitute teachers are typically under qualified to teach in the field of the regular 
classroom teacher, but are expected to fill the shoes of any regular teacher, whatever the subject matter field of the 
absent teacher (chemistry, mathematics, physics). 



In some school districts, the minimum education requirement level for substitute teaching is set at extremely low 
professional levels. For example, in the state of Utah, all a substitute needs to teach is a high school degree and no 
criminal record. Fairfax County, Va., demands only two years of college for its substitutes. (Streisand & Toch, 

1998). In recent studies, it was found that a large number of teacher-training graduates failed the recently introduced 
teacher-licensing exam. Last spring, nearly sixty percent of Massachusetts' education graduates failed a basic reading, 
writing, and subject matter test. A recent nationwide study by the Educational Testing Service found similar high rates 
of failure. (Streisand & Tock, 1998). 

It is also noteworthy that most substitute teachers are usually not equipped with lesson plans for the classroom periods 
that they cover and typically show a film or do other "filling time" classroom activities. In the substitute teacher chat 
room found on the internet, playing word bingo with the kids was a highly recommended form of filling class time. 

In essence, these substitute teachers are really not "substitutes" in an educational sense, but according to students 
highly paid "babysitters" to students that need quality teachers the most and require greater amounts of formal 
instructional time. 




This study will focus on teacher absenteeism at the high school level of instruction because this level is most directly 
associated with equal educational opportunity via AP courses, preparing students for post high school graduation 
education exit exams from high school, and general preparation for post secondary education. It is also the setting 
where students at one of the most important stages of their psychosocial development require time connections with 
their classroom teachers for purposes of modeling, receiving vision for a future, advice, and counseling. 

Finally, the role of geographical space on teacher absenteeism is often neglected as a variable in educational policy 
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studies. Geographical space has its own unique qualities with regard to understanding behaviors such as teacher 
absenteeism. For example there is 



• Location of the school setting: Where is the school located in geographical space? Where is the exact location 
and what is the land use surrounding the school setting i.e. the quality of the geographical space since it might 
impact on the quality of the instructional program, teacher turnover, and teacher absenteeism. 

• Livability of the geographical space surrounding the school: What is it like to live and be schooled in the 
geographical space of the school setting? Since most teachers don't live in the same geographical space as the 
school setting there is a tendency to ignore what it is like for students to live and study in the area. The social 
well being of the community, crime rate, extent of graffiti, etc. all impact on the livability of the geographical 
space and working conditions in the classroom. It also impacts on student preparation for class and attitudes 
towards school. 



• Likeability of school site area: How do other people in the community relate to their space and how is the 
pride in the community? Community well being, quality neighborhoods, and the support for schools are 
linked and have a major impact on the instructional program and what is expected from teachers and directly 
impacts on the motivation to teach. 

• Locus of Control: How does this geographical space dominate others in the area or is dominate with regard to 
quality? How are ideas, people, and goods moved into and out of the geographical space. What is the 
investment in the community, resident turnover, etc and how do others view the geographical space. 
Conditions where the geographical space is considered as negative implies that the space has little 
attractiveness for either living or working. This characteristic relates to teacher turnover and the sense of 
entitlement with regard to absence from the classroom. This characteristic of geographical space might affect 
teacher absenteeism and teacher turnover. 

This study combines recently developed data gathering mechanism on teacher absence rates at high school sites in a 
large urban school setting with Geographical Information Systems (G.I.S.). The purpose of this study is to map the 
association between the school's geographical space or physical environment (income level) and the propensity for 
teacher absence. The disparity between instructional resources as delivered vs. instructional resources as actually 
received by students in the classroom is examined in the context of a large urban school setting. 

Specifically, the purpose of this study is twofold. 

• First, to examine the geographical association between the quality of negative geographical space of a high 
school setting in a large urban school district (median family income in the area) and the rates of teacher 
absenteeism. 

• Second, to examine how teacher absenteeism measures and the need for substitute teachers (filled and 
unfilled) are associated with school performance at the school site. 

Methods 

This study used teacher absence information collected at all high schools in a large urban school district. The average 
absence per teacher was especially noted along with information regarding the number of unfilled substitute teacher 
positions. G.I.S. methods were then used to plot these school settings by those above and below a threshold 8.0 days 
of teacher absence per year per teacher at the school site. The quality of the geographical space surrounding the 
school setting or the median level of income for the zip code of the school setting was then applied to the map of 
school sites that were color-coded by high and low rates of teacher absenteeism. Finally, other important variables at 
the school site that might be related to substitute teachers, such as academic quality as measured by the Academic 
Performance Indicator or API, were also collected for each school site. 



In this study the following variables were collected at each high school site in the school district. 



Table 1 

Description of Variables 



Description 


Variable 


How often a teacher is absent from the classroom 


Teacher absenteeism rate 


Teachers without fully state certified credentials-many teaching in areas with no 
expertise 


Number w/o credential 



Teachers with less than 2 years of experience in the classroom- usually younger 
teachers 


Number < 2 yrs experience 


Number of requests to take the place of a classroom teacher 


Substitute teacher requests 


Number of times a substitute teacher cannot be found for the classroom 


Substitute requests unfilled 


Number of students leaving school and not entering any other educational 
program 


Drop out rate 


Number of students that leave before the end of the school year 


Transiency percent 


Number of times a students is asked to leave school for disciplinary reasons 


Number suspensions 


Number of times a students is asked to go to a different school- usually a 
continuation school 


Opportunity transfers 


Police reported crimes against property 


Crimes against property 


Police reported crimes against individuals 


Crimes against people 


Number of teachers needed to service student educational needs that are left 
unfilled 


Number of unfilled teaching 
position 


Level of academic performance at the school 


Academic Performance Index 



Findings 

Note in Table 2 that the average absenteeism for the school district was 8.27 days per teacher per year. Note that this 
figure ranges from a low of 6.3 to a high of 1 1 .4. Note also that the number of unfilled substitute teacher positions 
ranged from 9 to 306. The impact that geographical space might have on these statistics will be explored in a later 
section of the study. 



Table 2 

Descriptive Data for Senior High Schools in a Large Urban Area (n=49) 



Variable 


Mean 


St. Dev. 


Min. 


Max. 


Teacher absenteeism rate 


8.27 


1.06 


6.33 


11.44 


Number w/o credential 


26.84 


10.8 


8 


61 


Number < 2 yrs experience 


46.31 


14.6 


19 


89 


Substitute teacher requests 


1447 


590.5 


193 


2571 


Substitute requests unfilled 


85.39 


56.6 


9 


306 


Drop out rate 


5.53 


3.2 


.47 


15.7 


Transiency percent 


37.2 


12.4 


16.7 


77.1 


Number suspensions 


534.8 


345.2 


23 


1608 


Opportunity transfers 


76.22 


48.7 


2 


272 


Crimes against property 


11.27 


9.2 


2 


54 


Crimes against people 


78.24 


34.6 


37 


199 


Number of unfilled teaching position 


4.27 


5.1 


0 


21 


Academic Performance Index 


531.82 


83.3 


370 


737 



Note from Table 3, the high inter-correlation between teacher absence rates and academic performance (r = .54) and 
the amount of unfilled substitute teacher positions (r = .45) 

Table 3 

Inter-correlation of All Variables with Teacher Absenteeism Rates (**=p<.01) 



Variable 



Correlation 



Teacher absenteeism rate 


1.00 


Number w/o credential 


.37** 


Number < 2 yrs experience 


.24 


Substitute teacher requests 


-.01 


Substitute requests unfilled 


.45** 


Drop out rate 


.40** 


Transiency percent 


.50** 


Number suspensions 


-.04 


Opportunity transfers 


.19 


Crimes against property 


.44** 


Crimes against people 


.64** 


Number of unfilled teaching position 


,52** 


Academic Performance Index 


-.54** 



With the complete address for each high school site, the exact geographical location of a school site was identified to 
within five feet of its actual location. When the median family income for the zip code that contains the school site 
was added, a pictorial representation of the quality of geographical space and output indicators (especially teacher 
absenteeism) was visually depicted. 



Figure 1 depicts the large urban geographical area while Figure 2 is a plot of the location of school sites with teacher 
absenteeism above 8.00. Figure 3 is a plot of the location of school sites with teacher absenteeism rates both above 
and below 8.0, while Figure 4 is the same plot with the quality of the geographical space (median level of income) 
overlaid on the map. Figures 5 and 6 specifically examine school sites with above and below 8.0 teacher absence rates 
and Academic Performance Indices (API) above and below the 500 point average for the school district. 





Figure 2: Distribution of High Teacher Absence Schools (> 8.0) 




Figure 3: Distribution of High and Low Teacher Absenteeism Schools (Symbol X = highest and triangle = 

lowest) 
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Figure 4: Distribution of High and Low Teacher Absenteeism Schools with Geographical Space (Dark areas 
highest per capita income-(Symbol X = highest and triangle = lowest) 





Figure 5: Distribution of High Absenteeism-Poor Performance- Low Absenteeism-High Performance Schools 
(Symbol X = highest absence-poorest performance and triangle = lowest absence and highest performance) 




Figure 6: Distribution of High Absence- Poor Performing and Low Absence- High Performing School and 
Geographical Space (Dark Areas have Highest Per Capita lncome-(Symbol X = highest absence-poorest 
performance and triangle = lowest absence and highest performance) 

Table 4 

High Teacher Absenteeism Schools (n = 27) 



Variable 


Mean 


St. Dev. 


Min. 


Max. 


Teacher absenteeism rate 


9.01 


.77 


8.05 


11.44 


Number w/o credential 


29.22 


1.6 


11 


29.2 


Number < 2 yrs experience 


47.6 


14.2 


19 


47.6 


Substitute teacher requests 


1444.4 


637.1 


193 


2571 


Substitute requests unfilled 


99.2 


62.8 


15 


306 


Drop out rate 


6.1 


3.09 


1.25 


15.7 


Transiency percent 


39.8 


13.6 


18.42 


77.1 


Number suspensions 


628,5 


393.9 


59 


1606 


Opportunity transfers 


78.5 


59.8 


2 


272 


Crimes against property 


13.0 


11.2 


3 


54 


Crimes against people 


91.04 


38.6 


42 


199 


Number of unfilled teaching position 


5.75 


5.5 


0 


21 


Academic Performance Index 


505.74 


74.16 


370 


653 


Number of schools (n=27) 

Percent in Negative Geographical space (low community income)= 96% 



Note from Table 4 that high teacher absence (>8.0) was separated from those schools below 8.0. Note the variations 
in the API’s for these two groupings of schools (370-653) with an average of 505.- Also note that there is far more 
criminal activity and higher teacher turnover at these high absence schools 
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Table 5 

Low Teacher Absenteeism Schools (n = 22 ) 



Variable 


Mean 


St. Dev. 


Min. 


Max. 


Teacher absenteeism rate 


7.3 


.52 


6.3 


7.97 


Number w/o credential 


23.9 


10.5 


8 


49 


Number < 2 yrs experience 


44.8 


15.2 


22 


83 


Substitute teacher requests 


1451.1 


542.6 


402 


2508 


Substitute requests unfilled 


68.4 


51.9 


9 


237 


Drop out rate 


4.8 


3.3 


.47 


13.34 


Transiency percent 


34.1 


10.33 


16.74 


53.7 


Number suspensions 


419.8 


234.9 


23 


895 


Opportunity transfers 


53.5 


25.1 


2 


100 


Crimes against property 


9.2 


5.6 


2 


21 


Crimes against people 


62.6 


20.8 


37 


113 


Number of unfilled teaching position 


2.5 


3.9 


0 


18 


Academic Performance Index 


563.8 


84.2 


426 


737 


Number of schools (n=22) 

Percent in Negative Geographical space (low community income)^ 1 0% 



Note in Table 5 that low absence schools have an API of 426 to 737 with an average of 564. Also note that there we 
less unfilled substitute teacher positions. 

Table 6 

Inter-correlation of Variables with Teacher Absenteeism ** p <,01 * p <.05 



Variable 


High absenteeism 
Schools 


Low Absenteeism 
Schools 


Teacher absenteeism rate 


1.00 


1.00 


Number w/o credential 


.26 


.19 


Number < 2 yrs experience 


.24 


.31 


Substitute teacher requests 


.44** 


.31* 


Substitute requests unfilled 


1.00** 


.33* 


Drop out rate 


.21 


.45** 


Transiency percent 


.49** 


.31* 


Number suspensions 


.07 


.21 


Opportunity transfers 


.07 


-.02 


Crimes against property 


.61** 


-.16 


Crimes against people 


.51** 


.19 


Number of unfilled teaching position 


.52** 


.37* 


Academic Performance Index 


-.45** 


-.37* 


Number of schools 


27 


22 




Note from Table 6, the correlation with academic performance was -.45 and -.37 and the number teaching without 
teaching credentials .26 to -.19. 



Table 7 

High Teacher Absence-Low Academic Attainment Schools (n=16) 



Variable 


Mean 


St. Dev. 


Min. 


Max. 


Teacher absenteeism rate 


9.2 


.90 


8.1 


11.44 


Number w/o credential 


33.5 


10.2 


22 


61 


Number < 2 yrs experience 


52.4 


15.1 


19 


89 


Substitute teacher requests 


1465.1 


619.5 


477 


2571 


Substitute requests unfilled 


116 


67.4 


22 


306 


Drop out rate 


6.7 


2.9 


2.8 


15.7 


Transiency percent 


45.6 


14.1 


29.3 


77.1 


Number suspensions 


761.5 


451.5 


59 


1608 


Opportunity transfers 


82.81 


47.9 


2 


198 


Crimes against property 


15.8 


13.8 


3 


54 


Crimes against people 


110.2 


38.2 


62 


199 


Number of unfilled teaching position 


8.44 


5,7 


1 


21 


Academic Performance Index 


454,1 


36,7 


370 


497 


Number of schools (n~ 1 6) 

Percent in Negative Geographical space (low community income)= 100% 



Table 8 

Descriptive Analysis: Low Absenteeism-High Academic Performance Schools (n=16) 



Variable 


Mean 


St. Dev. 


Min. 


Max. 


Teacher absenteeism rate 


7,3 


,53 


6,3 


7.97 


Number w/o credential 


21,4 


10,2 


8 


49 


Number < 2 yrs experience 


89 


15.4 


22 


83 


Substitute teacher requests 


1482 


548,3 


584 


2508 


Substitute requests unfilled 


64.5 


55,2 


16 


237 


Drop out rate 


4,2 


2,1 


.47 


8.7 


Transiency percent 


31,1 


8,9 


16.74 


46.2 


Number suspensions 


382.7 


192,7 


23 


685 


Opportunity transfers 


47,4 


23,2 


2 


100 


Crimes against property 


8.3 


5.2 


2 


20 


Crimes against people 


57,1 


14.1 


38 


89 


Number of unfilled teaching position 


1.5 


2.0 


0 


7 


Academic Performance Index 


598.4 


71.0 


510 


737 


Number of schools (n= 1 6) 

Percent in Negative Geographical space (low community income)== 5% 




Table 9 

Correlations for High-Absence and Low-Absence Schools With Regard to Academic Performance 

(above and below 500 on the API) 



High teacher I Low teacher 




Variable 





absenteeism schools 
with low performance 


absenteeism schools 
with high performance 


Teacher absenteeism rate 


1.00 


1.00 


Number w/o credential 


.22* 


.19* 


Number < 2 yrs experience 


.09 


.41** 


Substitute teacher requests 


-.03 


.45** 


Substitute requests unfilled 


-.79** 


.40** 


Drop out rate 


.33* 


.36* 


Transiency percent 


.69** 


.31* 


Number suspensions 


-.10 


-.15 


Opportunity transfers 


.03 


-.10 


Crimes against property 


.69** 


-.10 


Crimes against people 


.76** 


.35* 


Number of unfilled teaching position 


.57** 


.52** 


Academic Performance Index 


-.88** 


-.33* 


Number of schools 


16 


16 



See Figures 5 and 6 for a geographical mapping of these schools. From Table 9 note the high correlation and 
differences in these correlation measures with regard to the number of teachers with less than 2 years of experience 
and the number of substitute teacher requests. Most important is the number of substitute teacher requests that remain 
unfilled and teacher absence. 



Policy Issues: The distribution of teacher absences 

The principal finding of this study is that the effect of teacher absenteeism is felt not equally across all school sites, 
but is felt most unfavorably in the urban schools or schools that are located in poor, low median family income 
geographical space. This finding corroborates the findings of other studies regarding how poverty impacts the urban 
school. As one national study has found, with teaching vacancies, rural and small town schools (10%) are less likely 
to use substitute teachers than central city schools (24%) and urban fringe schools (16%). TTiese rural and small town 
schools (96%) are more likely to hire qualified teachers than central city schools (90%); but they are almost equally 
likely to do so when compared to urban fringe schools (95%) (NEA, 1998). 

The amount of teacher absences at schools in these negative geographical space areas is also an indicator of teacher 
morale and stress. When there is high teacher absence, it tends to lower the morale of remaining teachers resulting in 
higher teacher turnover. This could be the cause of the failure of these schools to attract more committed teachers or 
the danger and stress posed by working in high-risk neighborhoods that result in teachers leaving the classroom. 

Since urban districts are often plagued with gang violence and unsafe schools and neighborhoods, the latter condition 
is highly significant to examine for the policy maker since there is an actual threat to a teacher’s physical safety. As a 
result and as expected, there should not only be more teacher absence in poorer areas of the city, but lower retention 
rates of qualified and experienced teachers. 



Summary, Recommendation, and Conclusions 



As noted earlier, students in a classroom eventually lose the desire to learn when the regular teacher is frequently 
absent and the delivery of the instructional program is by an array of substitute teachers. In essence, teacher 
absenteeism has important implications for school reform, teacher training, educational leadership, and issues related 
to social justice aimed at promoting equity and excellence in the schools. The use of substitute teachers puts added 
strain on the efforts of school reform by not only increasing the costs of instruction, but by changing the delivery of 
instruction from fully credentialed and permanent teachers to part time and substitute teachers. In addition with heavy 
reliance on substitute teachers to deliver instructional programs, the teaching efforts of the regular teacher towards 
school reform might be undermined. One of the main characteristics between successful and unsuccessful school 
reform efforts is the closing the disparity between resources as distributed vs. resources as received in the classroom. 
The degree of teacher absence at a school site is directly related to this resource distribution-reception disparity. 

Finally educational leadership has to recognize the fact that teaching at low-income area schools increases the 



propensity of teachers to be absent. With this in mind highly trained substitute teacher pools might need to be 
developed at the site so that instruction is continued when the teacher is absent. The shortage of substitute teachers 
(Lilly, 1998) and the effective use of substitute teachers (Holdaway and Benhaw (1974) in the classroom have also 
been studied and should be considered part of school administrative training. (Pitkoff, 1993). Some states 
implementing large scale studies of substitute teaches include Wisconsin (Substitute Teaching in Wisconsin, 2000) 
amd Nova Scotia (Unicomb et al., 2000) 

In conclusion research on low-income schools should examine in closer detail the disparity between resources as 
delivered to the classroom vs. resources as actually received by students in the classroom. Teacher absenteeism is a 
major contributor to instructional resource disparity or resources that are not actually being delivered to students in 
the classroom. In addition the notion of attenuation or the lessening of the impact of school resources and the 
amplification of risk factors of students needs to be examined in much closer detail by educational researchers. 

This study partially demonstrates that the dual problem of attenuation of school district resources and amplification of 
student risk are both impacted by teacher absenteeism. In addition, the quality of geographical space as measured by 
the median income of the area is also highly associated with teacher absenteeism. This association between the quality 
of geographical space, teacher absenteeism, and educational attainment raises issues of social justice and has the 
potential to undermine school reform efforts at promoting equity and excellence in urban schools. 
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Abstract 

Since the late 1 970s the problem of urban education has been cast as partially a problem of 
governance and authority structures. This focus mirrors a larger preoccupation by educational 
reformers with democratizing the decision-making process in public schools, a preoccupation 
that is evident not only in this country but also many nations throughout the world. Borrowing 
from the private sector, the underlying assumption behind decentralization is that educational 
improvement is only possible if those closest to the point at which decision are enacted become 
the architects of these decisions. Thus, school-based management or participatory decision- 
making is viewed as a means to formally incorporate the voices of parents, teachers and the 
community in the management of their schools. This paper discusses the findings of a recently 
conducted study on school-based management in thirty of New Jersey's poorest districts (referred 
to as the Abbott Districts). These districts have begun a process of complex reform after the 
State’s Supreme Court ruled that the state had failed to constitutionally provide a thorough and 
efficient education for its poorest students by the absence of parity funding. Populated by 
primarily black and Hispanic students, and representing most of the larger urban communities in 
the state, students in these districts exhibit performance levels significantly below that of the state 
average. The results of the study indicate that (1) genuine autonomy has been usurped by an 
intensification in state power and authority, (ii) state elites have provided little opportunity for 
districts and SBM teams to build capacity; (iii) the level of democratization or opening-up of 
decision making to local community members has been minimal as the teams become teacher 
dominated; and (iv) in the absence of clear guidelines from the State, conflict over the 
appropriate role of SBM members, principals, central office staff and local school boards has 
emerged. The paper on the basis of these findings explores some policy options that need to be 
considered both at the state and local levels as school communities move toward more 
decentralized governance structures. 
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Introduction 




Education remains one of the primary means through which social mobility is attained. Yet, the many discourses on 
the state of educational institutions suggest institutions that are imperiled for a variety of reasons. This crisis in public 
education is viewed as more pronounced in communities peopled by the poor of Latino and African- American 
descents than in white affluent communities. While the problems of these educational systems have been framed in 
many different ways, one argument that has been consistently forwarded centers on the endemic paralysis of their 
central bureaucratic structures in responding to efforts of change. Consequently, a popular policy solution has focused 
on the devolution of power and authority from these central bureaucracies to less formal and rigid structures i.e. 
schools. However, the history of the decentralization movement reveals noticeable ideological shifts behind the 
purpose of school-based management (SBM). 

During the sixties, attempts were made to increase the level of participation in decision-making through the formal 
incorporation of various subgroups. Concerned with such issues as granting greater power and authority to local 
communities, diffusing state authority and increasing organizational efficiency, the decentralization movements of this 
era saw the devolution of authority as a means of meeting political and administrative ends (David, 1989; Wohlstetter 
& Mohrman, 1996). The eighties however, witnessed a change in the purposive intent behind decentralization. This 
change resulted from the pervasive influence of the reform movements that dominated the educational landscape of this 
period. During the eighties, there was a broad call for the implementation of comprehensive educational changes - 
changes that addressed professional development and instruction, the replacement of bureaucratic regulations with 
professional responsibility and accountability, and the development of high standards for teachers as well as students 
(David, 1989). The focus of school-based management thus became inextricably interwoven with concerns about 
student achievement. 

The growing popularity of school-based management as a reform strategy is evidenced by the fact that in 1 993 over 44 
states practiced some form of decentralized governance (Herman & Herman, 1 993). Within the broader global context, 
decentralization became an integral component of the reform movements in countries such as New Zealand, Canada, 
Britain, Spain, and Wales (Hanson & Ulrich, 1994; Leithwood & Menzies, 1998). Ironically, at the same time that 
increasing numbers of school districts, states, and nations were adopting decentralization policies in the hope of 
bringing about improvement in student achievement, the evidence suggested that school-based management was less 
powerful a source of school improvement than its advocates believed. Indeed, the evidence continues to show that the 
impact of school-based management is more apparent in the areas of governance and organizational structures than in 
changed classroom practices and improved student achievement (Summers & Johnson, 1991; Wohlstetter & Mohrman 
1996). 

Notwithstanding this trend, several arguments have been advanced by proponents of decentralization in support of the 
superiority of this form of governance arrangement over centralized structures (Murphy, 1991). First, it is argued that 
decentralization gives communities, parents and teachers a stake in local educational decision-making. Second, 
decentralization is seen to contribute to the evolution of greater levels of professional commitment by allowing 
teachers to exercise a voice in decision-making. Third, the suggestion is proffered that the creation of decisions at 
levels that are closest to students, results in better outcomes, as those making the decisions are more acutely aware of 
the needs of these students. Fourth, decentralization is viewed as a mechanism that has the potential to promote greater 
efficiency in the utilization and expenditure of resources. This is achieved, since the decisions are being made by those 
closest to the point where services are being delivered, thereby resulting in a greater match of services to needs. Fifth, 
since bureaucracies are perceived to be ineffective in meeting the needs of students, decentralized structures are 
considered to have the potential to be more responsive to student needs than are bureaucratic organizational forms. 

The Theoretical Underpinnings of School-Based Management 

The above arguments on the advantages of this form of school governance reveal some important theoretical 
assumptions. Undoubtedly, the notion of decentralization in educational decision-making and governance issues 
appeals to the social democratic principles of egalitarianism whereby local communities acquire a voice in institutional 
building and operation (Seddon, Angus & Poole, 1990). If this principle is actualized through the creation of 
democratic decision-making structures, a significant shift in the realignment of power relationships can be expected to 
occur. Specifically, grass root groups functioning in some combination with school-based leadership ideally would 
replace the dominance enjoyed by educational bureaucratic elites in local school governance matters. Devolution of 
authority thus enables the education constituency to become more inclusive and less narrowly restricted to technocrats. 
By accomplishing this, the balance of power between educational elites on the one hand, and local community and 
school-based actors on the other, is redistributed to the advantage of the latter group (Seddon, Angus & Poole, 1 990). 

Theoretically, this redistribution of power corresponds to a re-conceptualization of the organizational unit deemed to be 
most important administratively for the improvement of learning. Under the old governance model, central office units 
were considered to have the administrative responsibility for ensuring that the conditions needed to promote learning 
were in place. With decentralization, the school as a subunit now assumes this role. 

Organizational and economic arguments have also played a role in framing some of the assumptions on which the 
concept of decentralization of authority structures in educational settings is grounded. Some organizational theorists 
argue that a decentralized environment is optimal for efficiency in operations, since employees who are empowered to 
make decisions have more control over their work and hence become more accountable for decisions (Murphy, 1991). 
The premise of these arguments is that by flattening the decision-making process, and bringing it closer to the site 




where client needs are met, the effectiveness of the organization is improved, as employees based on their knowledge 
and interactions with clients can reshape their products and services based on an understanding of client needs. 

Miron (1996) posits to the contrary however, that the incorporation of corporate principles of decentralization as 
reflected in the ideology of shared decision- making ought to be approached with caution by educators. According to 
Miron, corporate downsizing and decentralization of decision-making represented a strategic response by capital to the 
global fiscal crisis. However, the relative complexity of schools’ institutional processes when compared to those in the 
corporate world implies that the importation of the ‘logic of capital' into educational institutions can create a set of 
discursive practices, as well as mask some of the macrostructural and micropolitical processes that are in play. 



In a similar vein. Ball and Smyth have advanced a critical political-economic perspective on school-based management 
(Ball, 1993; Smyth, 1993). Both have advanced the notion that decentralization ought to be understood within the 
context of resource availability, social responsibility, and accountability. From these writers' perspectives, the social 
democratic principles on which decentralization is premised, and which appear appealing to constituencies whose 
voices have been rendered mute by educational elites, belie some of the hidden motivation behind those at state and 
governmental levels who push this form of governance. Specifically, the argument is posited that the devolution of 
authority from central sources, especially at the state level, serves to legitimize state agencies in many ways. First, it 
gives the appearance that these agencies are sensitive to local needs. Second, by shifting decision-making 
responsibilities to the schools, these agencies can distance themselves from failed policies by blaming schools for poor 
management and flawed decision-making. This works in the favor of state elites by insulating them from the 
consequences and contradictions that are generated by the formulation of poor policies. 

Moreover, both Ball and Smyth view the devolution of authority to local schools as placing unfair burdens on schools 
in instances of resource scarcity. Under these conditions, schools are placed in the unenviable position of having to 
make decisions on how to distribute scarce resources. However, in doing so, decentralization serves an important 
conflict management function. Welier's refinement of the latent functions inhering in decentralization amplifies this 
underlying thesis of the political-economy perspective (Weiler, 1990). Weiler argues that decentralization has two 
latent functions: one that serves to legitimize certain socio-political arrangements, the other that allows for the 
management of conflict. Welier suggests that in policy contexts that are potentially highly conflictual, such as 
education policy arenas, decentralization is politically instrumental in helping to diffuse and manage conflict (See also 
Anderson & Dixon, 1993; Seddon, Angus & Poole, 1990). 

The opposing theoretical arguments that have been presented in this paper imply that decentralization is far more 
complex in its implications for schools than is popularly understood. Not-with-standing the problems that are 
associated with highly centralized structures, the lack of any substantive data on the significant impact of decentralized 
forms of educational governance on student achievement coupled with the problems that have been encountered with 
the decentralization movement, suggest that closer intellectual scrutiny of this concept is warranted (Anderson & 

Dixon, 1993; Gordon, 1992; Wohlstetter & Odden, 1992). 

Purpose of Study 

In 1998, in its culminating decision on the legal challenges to the State of New Jersey's funding and educational 
policies with respect to the state's poorest districts, the New Jersey Supreme Court ordered the implementation of a 
series of remedial measures aimed at redressing the long standing disparities between poor and affluent school districts. 
The decision referred to as Abbott V, sets out an ambitious agenda for reform that includes changes in instructional 
programming through the adoption of whole school reform models, expansion of early childhood programming and 
school- community social agency linkages, as well as improvement in facilities {Abbott v. Burke, 1998). 

According to regulations published by the New Jersey Department of Education, the process of implementing the 
reforms ordered by the State's Supreme Court is to be guided and led by teachers, parents, community and other school 
level staff through the formal establishment of school management teams. The regulation states that the purpose of 
these teams is to “ensure participation of staff, parents and the community in school level decision making and to 
develop a culture of cooperation, accountability and commitment” (New Jersey Department of Education, 2000). To 
that end, the school management teams are expected to guide and lead decisions on curricular, instructional, personnel 
and budgetary matters. 

This study in light of the preceding discussion on the unresolved theoretical and empirical issues plaguing the notion of 
participatory decision-making, as well as New Jersey's current policy guidelines governing the implementation of 
decentralization, raises and seeks to answer the following questions with respect to school-based management in the 
state's thirty poorest districts. First, what is the level of democratization that has occurred in these systems? This 
question is answered through the posing of two related concerns; the extent to which participation in decision-making 
reflects the major constituencies that are intended to be on the school-management teams; and the degree to which the 
process allows for the legitimate exercise of decision-making and authority. Second, how has school-based 
management resulted in the successful devolution of authority from centralized to decentralized localities? The 
questions as they are posed, speak more to the issues of whether school-based management in its empirical form is 
consonant with the assumptions of democratizing decision-making and hence the social-democratic principle of 
egalitarianism and less with the effects of this form of governance on student achievement. 



Method 



Sample, Instrumentation and Data Collection 

This study employs a mixed method research design. In April 2000, a questionnaire was mailed to a randomly drawn 
sample of 140 elementary and middle schools' school management teams. Included in the survey were questions on 
team membership and composition, the extent to which factors identified as germane to a team's ability to function, 
such as training, and group coalescence influenced the legitimate exercise of decision-making and the quality of 
support provided to the teams by the State Department of Education. The survey elicited a response rate of 5 1%. The 
school management teams in the study represented four different cohorts of schools. These cohorts correspond to the 
timeframe in which the schools begun to implement whole school reform. 



According to state regulations, schools had three years within which to begin their whole school reform process. 
Schools that started the process within the first year of the Court's decision were referred to as Cohort I schools. 
Similarly, schools that begun in the second year were designated as Cohort 2 schools, schools during the middle of the 
second year mid-year cohort schools, and during the third year, Cohort 3 schools. In our sample there were 1 5 Cohort 1 
school teams, 14 teams representing Cohort 2 schools, 6 teams from mid-year Cohort schools, and 32 teams belonging 
to the third Cohort of schools. Five teams failed to identify their cohort status. Knowing the cohort status of the team is 
important to the study at hand, since Cohort 1 school management teams- that is teams belonging to schools who 
started the reform process a year after the Court rendered its decision in 1998- had very little time to engage in quality 
planning. 



In addition to surveying the school management teams, two focus groups were held. The purpose of both focus groups 
was to gain an understanding of the processes that were involved as authority got devolved from the central offices to 
the schools. The first focus group was held with one central office representative from six school districts. These 
districts were chosen to reflect the racial composition of their student bodies, their geographical locations in the State, 
their governance structures and when they were classified as being an “Abbott District”. A second, less formally 
structured focus group discussion was held with three superintendents in October of the same year. These 
superintendents were executive members of the statewide association of urban superintendents. 

Data Analysis 

The data analysis involves the use of descriptive statistics and the statistical testing of associational relationships, 
through the use of Chi Square and Analysis of Variance. Standardized residuals are reported when significant chi- 
square values were found. These residuals allow us to identify the categories that are making a significant contribution 
to the significant chi square value. Following Haberman's guideline, it was inferred that where the standardized 
residual for a category is greater than 2, that category is strongly contributing to the significant chi square value 
(Haberman, 1 984). Tukey post hoc testing was done for those Anovas that were found to be significant. Data gathered 
from the focus groups data was subjected to a qualitative analysis. 

Findings 

Degree of democratization of school-based management in the Abbott Districts 

The New Jersey Department of Education guidelines state that the constituent groups that must be represented on the 
school management teams are the building principals, teachers, school-level support staff, parents, and community. The 
inclusion of students is an optional requirement that is left to the discretion of the individual school. Groups or 
individuals excluded from membership on a team are Board of Education members and district employees who wish to 
serve in the capacity of a parent or community representative. According to the regulations, no one group can 
constitute 50 percent or more of a team's total membership. Membership on a team is secured either through an 
electoral process or by selection. The minimum number of years that a given member can serve on a team is two, 
however, to ensure continuity in the event of an election or selection, teams are allowed to stagger membership. 



Murphy and Beck (1995) suggest that school based management typically assumes one of three ideal forms; 
administrative control SBM (in this model the principal is the primary decision maker), professional control (teachers 
are the primary decision makers) and community/parent control (community members and parents comprise the major 
decision making groups). A fourth though less popular form is defined by Malen and Ogawa as balanced control 
(Hanson & Ulrich, 1994; Malen & Ogawa, 1988). In this model an attempt is made to establish a balance in decision- 
making among all stakeholders. Within the context of New Jersey, it is clear from the regulations that the Department 
of Education promulgated that the attempt was to create a model that approximated a balanced control form of SBM. 
The guidelines stated that the model to be adopted by schools was one, which restricted the membership of any given 
stakeholder group to less than 50 percent of the total membership. 

In actuality however, the findings from the present study indicate that SBM in New Jersey is regressing towards a 
teacher-dominated form of SBM. Of the sixty-nine teams with valid responses on membership composition there were 
^ 17 teams in which the teaching staff members represented more than 50% of the total membership and 1 3 teams on 

ERIC 




which teachers made up half or 50% of the total membership. Thus, 43% of the teams had at least half of their 
membership drawn from the teaching staff. The dominance of teachers on the school management teams cut across all 
cohorts. However, proportionately more of the teams that were dominated by teachers were apt to be in schools 
belonging to the first cohort. 

With respect to representation from other stakeholders, while more than 90 percent of the teams reported having at 
least one parent member, about 26 or approximately 38% of the teams were at the time of the study without community 
representation. The twenty-six teams reporting no community presence were proportionately distributed among the 
various cohorts, although slightly more 43% or 6 out of the 1 4 second year cohort teams in the study indicated that they 
had no community representation. On the other hand, only 7 teams had no in school -support staff representation. The 
data provided by the teams in the study reveals that most teams lacked student representation. Indeed 58 teams or 
roughly 83% of the teams reported that there was no student membership. 

In examining the degree of representation of the major constituencies on the teams, the proportions for each group were 
calculated on the basis of the size of the team. On the whole the median proportion for teachers was .47, for parents .22 
and for community members. 07. This implies that on half of the teams, teachers made up 47% or more of the teams* 
membership, parents 22% and community stakeholders 7%. In-school support staff, and school administrators 
constitute the remaining percentages. Parent and community groups thus accounted for about 29% of the total 
memberships, while seventy-one percent of the teams' membership are drawn from school -based personnel. These 
findings suggest that the evolution of school-based management in the Abbott districts has resulted in some instances, 
in structures, which deviate from what was originally intended. The balanced model, which was initially proposed, has 
not been the dominant form. 

Whether or not, school-based management has successfully resulted in democratizing the process of decision-making 
by incorporating the voices of key constituent groups remains therefore questionable in light of these findings. Even in 
those instances in which parental participation is secured, the dominance of school-based personnel has overshadowed 
the voices of parents. Kildow's case study of one team's functioning described how the parent member frequently 
deferred to school-based members on all issues, and viewed herself as less empowered to make decisions when 
compared to her school-based counterparts (Kildow, 2000). What these findings seem to suggest is that the ‘social 
empowerment' of parents and communities that proponents of this form of governance arrangement imply is attendant 
with participatory decision making has not occurred in the New Jersey reforms. 

Barriers to the legitimate exercise of decision-making 

The primary responsibility of the teams is to develop a plan that will guide the school's implementation of its whole 
school reform model. The teams are also responsible for ensuring that their schools' curriculum and instruction are 
aligned to the New Jersey Core Curriculum Content Standards. They are expected to engage in a needs assessment 
process based on a review of student performance data on the statewide assessments and on the basis of this review 
make recommendations for curricular and instmctional improvement. Teams are also required to ensure that there is a 
program of professional development for teachers in their individual schools linked to the school's whole school reform 
model. Each school is further responsible for the development of a technology plan that is submitted to the Department 
of Education for approval. In addition to these responsibilities, the teams are also expected to ensure that there are 
programs and activities that are linked to the cross content readiness standards in the core curriculum standards, as well 
as develop a school based reward system for teachers, administrators and parents who contribute to students 
successfully meeting these standards. Finally, the teams based on a majority vote and with state department approval 
(through the School Review and Improvement Team) are responsible for approving a school budget and may 
recommend the appointment of a building principal, teaching staff member and instructional aides. 

The teams were asked to rate their abilities to function effectively along several operational dimensions that previous 
literature has identified to be important influences on a team's capacity to successfully govern. These dimensions 
include: clarity about roles and responsibilities, membership commitment, understanding of a shared mission, meeting 
schedules, attendance at meetings, effectiveness in communicating with the larger school community and active as 
opposed to token participation in decision - making. Overall, the teams in the present study exhibited ambivalence in 
their evaluation of their abilities to effectively govern. Teams were unanimous that their membership was committed 
(86%) and that individual interests did not supersede the goals and mission of their work (88%). Neither did teams 
report that conflict among members posed a barrier to their ability to operate effectively (91 %). Indeed, ninety percent 
of the teams reponed that they were able to deal constructively with differences in opinions among them.selves when 
these differences arose. However, when an examination of the association between team composition and the 
identification of barriers that impede the teams' abilities to function effectively was done, some interesting findings 
emerged. 

Teams that lacked community representation were more likely to indicate that individual members' self interests took 
precedence over team matters. A chi-square value of 8.75 was found to be significant, and the standardized residuals 
showed values of 2 or greater for teams with poor community representation and the identification of problems with 
individual self-interests. Also, teams with no community representation indicated that they were less likely to explore 
alternatives when making decisions than teams with community representation (Chi-Square value of 8. 1 18 was found 
to be significant at the .044 level). Again, standardized residuals were larger for these teams. On the other hand, teams 
without community representation were less likely to report problems with attendance at meetings than those with 




community representations (Chi Square value of 6. 109 was found to be significant at the .05 level). The data also 
showed, that teams who were cajoled to start their whole school reform process early, that is cohort 1 teams, were 
significantly more likely to report problems with commitment, than those teams that started the process much later (Chi 
Square value of 9.456 was found to be significant at .045 level). 

Table 1 

Association between Community Representation, 

Cohort Status and Factors Impacting a 
Team’s Ability to Successfully Govern 



Relationships 


i} 

Values 


Significance 

Level 


Community representation and Problems with individual interests taking precedence 
over team matters 


9.640 


.01 


Community representation and teams exploration of alternatives when making 
decisions 


8.118 


.04 


Community representation and attendance at meetings 


6.109 


.05 


Cohort status and members commitment 


9.456 


.05 



Note: Total number of teams in analyses of community representation: 69; number of teams in 
cohort analyses: 47. 

With respect to role clarity, about one-third of the teams (31%) indicated that they were unclear as 
to their roles and responsibilities. About the same percentage (33%) also reported difficulties in 
communicating with their larger school communities. Securing adequate involvement from all 
potential constituent groups was raised as another problem area affecting the ability to govern. The 
experiences of teams in the larger school districts are instructive on this issue. According to these 
teams, the restrictive clause in the regulations which preclude in-district employees from serving in 
the capacity of a parent or community representative has hampered their abilities to recruit 
membership, as a significant number of local residents have an employment status with the school 
system 

Lawler (1986) argued that legitimate participation has four requirements: knowledge and skills, 
power, information, and rewards. This framework has been used by Wohlestetter et.al (1994) to 
explain variations in implementation and effects among SMTs operating in different contexts. In 
surveying the teams in the Abbott districts attention was paid to three of these requirements, 
knowledge, skills and information. Teams were asked to rate on several scales their level of 
knowledge, previous experience and comfort in the ten areas of their responsibilities. It is reasonable 
to assume that the experiences, which members on the school management teams bring to their new 
roles are likely to impact qualitatively on the kinds of decisions that are made, and the teams comfort 
in doing so. Data on the number of team members who have had prior experiences in the 10 areas 
for which they are responsible indicate that overall very few teams are composed of members who 
have had prior involvement in any of these areas. As can be seen in Table 2, experience is weakest in 
the areas of school-based budgeting, technology planning, school-based hiring decisions and 
developing reward systems. Teams had proportionately more members, who prior to joining the 
teams had some experience with curriculum alignment and needs assessment. 

Table 2 

Percent of SMT Members with Prior Experience in 
Each Area of Teams’ Responsibilities 



Areas of Responsibility 


Percent of Members with Prior Experience 


Aligning Curriculum 


41.7% 


Conducting Needs Assessment 


37.1% 



Working on, or reviewing professional development programs 


32.2% 


Involved in developing school-based reward systems 


30.0% 


Involved in school-based hiring decisions 


19.9% 


Worked on developing a technology plan 


16.8% 


Involved in school-based budgeting (zero-based) budgeting decisions 


9.8% 



Number of teams responding: 66 

It is quite conceivable, that although Team members may lack the experience base for making 
decisions, that nevertheless, they may have an informed knowledge base that can be drawn upon in 
decision-making situations. Each Team was asked to indicate the degree of knowledge it possessed as 
an entity in each of the 10 areas of responsibility. These responses are summarized in Table 3. 

About one third of the Teams felt that their knowledge base on how to align curriculum, review test 
score data and determine program and curricular needs on the basis of this review, as well as 
determine what actions need to be taken to improve academic achievement in their schools was 
substantive. On the other hand, a significant proportion (over 75%) felt that they had only some or 
no knowledge on how to 1) develop a professional development program that is related to the 
implementation of the reform, (2) develop a technology plan, 3) make decisions with regard to hiring 
school personnel, 4) develop a school-based budget and 5) develop school based reward systems. 
Significant differences were found among the cohorts. Teams belonging to the first cohort were 
more likely to report lack of knowledge with respect to developing school-based budgets than teams 
belonging to the second, mid-year and third year cohorts. Third-year cohort teams were also more 
likely to report having less knowledge on creating professional development programs than the 
second year cohorts. 



Table 3 

Areas of Responsibilities: Percent of Teams Reporting Minimal Knowledge 



Areas of SMT Responsibility 


Percent of Teams Reporting Minimal 
Knowledge 


Aligning Instruction to the Core Content Standard 


55.4% 


Deciding what actions needed to be taken on the basis of test score data 


59.1% 


Reviewing test score data as part of a needs assessment process 


62.1% 


Determining program needs on the basis of test score reviews 


66.7% 


Making curricular decisions on the basis of test score data 


66.7% 


Developing school-based reward systems 


75.8% 


Developing a professional development program that is linked to the 
implementation of the reforms 


79.4% 


Making school-based personnel decisions with respect to hiring 


80.3% 


Developing a technology plan 


80.6% 


Developing a school -based budget based on zero-based budgeting 
procedures 


82.1% 



Training is a critical component in the development of the knowledge and capacity of teams to 
function effectively in making quality decisions. To that end, teams were asked to rate the adequacy 
of training they received around the major substantive areas for which they have responsibilities. 
Twenty-three teams reported that they received no training around any of the areas for which were 
given responsibility. Overall, the teams who provided feedback, were more favorable in their ratings 
of the training received in areas related to curriculum, test score analysis and school-based 
professional development, than they were in their evaluation of the training provided around school- 
based hiring decisions and developing school-based reward systems (see Table 4). 




Table 4 

112 



Percent of Teams Rating Training Received to be at Least Adequate 



Area of Training Support 


Percent of Teams Rating Training to be at 
Least Adequate 


Roles and responsibilities of the teams 


59.0% 


Developing acceptable standards for professional development 


56.8% 


Curriculum Alignment 


53.2% 


Use of test scores for decision-making 


51.1% 


Analysis of test scores 


50.0% 


Technology planning 


38.9% 


Developing school-based reward systems 


28.6% 


Hiring procedures for school-based personnel 


24.5% 


Developing school-based budgets 


21.1% 



Number of teams responding: 47 

Given the fact that teams lack the knowledge and experience to adequately fulfill their 
responsibilities, and given the unevenness in their satisfaction with the training that they have 
received, how comfortable are the teams in making the decisions that are expected of them? Data 
provided by the teams in the survey indicate that teams feel more comfortable in making decisions 
related to curricular and instructional issues than they do in making decisions that involve 
technology, school-based budgets, school hiring decisions and reward structures. For example, more 
than sixty percent of the teams reported that they are uncomfortable in creating rewards for 
teachers, and more than 80% indicate that they would be similarly uncomfortable in determining 
rewards for their building administrators. Forty-percent of the teams indicated that they would not 
be comfortable in making decisions involving the hiring of a principal and a similar percent 44®/o 
expressed discomfort in making teaching appointments. 

In some of these decisions making areas teams are required to vote on whether or not they wish to 
have input. At the time of the survey, only 21% of the teams had voted to provide input into the 
hiring of their building principal and 26 % for input into the appointment of instructional aides. 

Data culled from the focus group discussion reinforced the notions that some teams are reluctant to 
get involved in hiring and budgeting decisions. According to the central office administrators in the 
focus group, while some teams initially wanted to select personnel for their buildings, they 
experienced discomfort when the process of selection begun, especially in those instances when they 
had to make decisions about staff on their own level. These results parallel similar findings reported 
by Jones') study of teacher decision-making preferences in Texas (Jones, 1997). Jones found that 
teachers expressed a desire and were more involved in areas concerning curriculum/instruction and 
student services than staff, personnel and budget management. 

Decentralization provides the impetus for the creation of a new institutional culture within schools. 

It also presupposes that some socialization occurs whereby all actors are socialized to their new roles 
and responsibilities. However, our discussion so far suggests that the exercise of legitimate decision- 
making has been constrained by the teams' inexperience, uneven knowledge base, and the absence of 
adequate training to build capacity. 

Devolving power from central to decentralized structures: Decreasing autonomy or increasing centralization? 

The New Jersey Department of Education has created a structure, the School Review and 
Improvement Team (SRI) that ostensibly functions in the capacity of an overseer of the reform 
process, ensuring that the implementation of SBM is progressing according to the guidelines set forth 
in the regulations. The School Review and Improvement Team is comprised of Department of 
Education personnel from the Divisions of Student Services and Finance. Each school in an Abbott 
district is assigned to a team that is based at one of the State's Program Improvement and Regional 
Centers. The SRI Teams have a wide range of responsibilities to include working with the districts 
and building principal to ensure the effective implementation of whole school reform and school- 



based management; consulting with the school management teams to ensure that all of the SMT 
responsibilities are effectively fulfilled; serving as liaisons between the schools and the Whole School 
Reform model developers, and consulting with the Superintendents on the transfer or removal of 
teachers and principals. 

There are two related issues that one may surface regarding the balance in power between the State 
and the local sites in the reform process. First, according to David (1989), a policy cornerstone of 
successful decentralization involves the accompanying of local autonomy with simultaneous relief 
from onerous rules and regulations (See also Herman & Herman, 1992, Hill & Bonan, 1991). The 
extensive regulatory role played by the School Review and Improvement Team in the 
decentralization process in New Jersey seems to stand in contradistinction to David's observation. In 
fact, the question can be posed as to whether or not the regulations governing the role of the SRI 
teams have the potential to undermine local autonomy and thereby result in an intensification of 
power at the State level, rather than a real gain of power at the school level? The strong regulatory 
presence of the Department of Education through the School Review and Improvement Teams far 
exceeds and is different from the decentralization and centralization tendencies that many state 
reform strategies have exhibited (Boyd, 1992; Levacic, 1995; Levin, 1997). 

These strategies evident in other reform efforts have combined shifts in authority to local schools 
with state control over setting and monitoring standards. However, the School Review and 
Improvement Teams* roles extend beyond one that is primarily of a monitoring nature. The SRI 
among other responsibilities approves decisions made by the local schools, decides when a team can 
assume new responsibilities in the areas of budgeting and personnel (if teams decide by a majority 
vote to assume these responsibilities) and approves transfers or firing of principals and teachers. In 
effect they have assumed an external governance role thereby adding another bureaucratic layer to 
the reform. One may argue that the SRI structure, which the Department of Education has put in 
place to provide field-based assistance to the schools and their respective management teams, 
virtually places the Department of Education in the position of assuming responsibility for the 
success of the reforms. Thus, the NJDOE may not be able to distance itself from any failed policies 
associated with the reforms. 

This broad notion of shared responsibility that is being advocated here implies that state, local 
districts, and schools are equally contributing to the successful implementation of the reforms. Since 
the SRI is the primary state resource that is being directed to support the schools, the question as to 
how effective this field assistance has been is relevant to raise. The School Management Teams in the 
study were asked to indicate their degree of satisfaction with the support provided by the SRI teams 
in the areas stipulated by the regulations. The following discussion presents the Teams responses. At 
the time of the survey more than one third of the teams had not yet had a meeting with their SRI 
facilitator. Furthermore, several of the teams were unfamiliar with the roles and responsibilities of 
the SRI and sought clarification from the researchers. Thus only 41 of the 72 teams were able to 
provide feedback on the SRI teams. Among the districts providing feedback, there was a high level 
of dissatisfaction with the support that the SRI teams have provided. Seventy- one percent of the 
school management teams reported that their SRI facilitator attended meetings irregularly, and 56% 
noted that the technical assistance provided was unsatisfactory. 

While, about 56% of the school management teams stated that their SRI provided assistance with 
general implementation issues, 54% noted that the SRI teams provided no assistance with the actual 
development of their implementation plans. Furthermore, more that 58% of the teams were 
dissatisfied with the help received from their SRI Teams with problems encountered during 
implementation; and an even larger percent 68% indicated that their SRI team provided minimal 
assistance in working with the model developers. An equally substantial number of the teams (25 or 
68%) noted that their SRI did not help in identifying areas for training, neither were the SRI 
facilitators helpful in assisting them in the identification of experts that can help with the problem of 
student achievement. With respect to the budgeting process, more than 61% of the school 
management teams reported that they were dissatisfied (or unsure of how satisfied they were) with 
the assistance, which their SRI facilitator provided in the development of the school budgets. 

Overall, only about 38% of the teams reported general satisfaction with the support, which they have 
received from the SRI Team that has been assigned to them. 

When cohort status is entered as the main effect in several one-way ANOVAS in which evaluations of 
the SRI various responsibilities are treated as the dependent measures, several significant findings 
were found. According to the data furnished in Table 5, the impact of cohort status on the teams* 
evaluation of the SRIs was significant in six areas. (See Table 6) These were: help in 



implementation, providing satisfactory technical assistance, providing assistance with the school's 
implementation plan, helping with the model developers of the various whole school reform models, 
providing assistance in school-based budgeting and overall support. Results of the Tukey post hoc 
testing reveals that schools belonging to the first cohort were significantly more dissatisfied with the 
support, which they received from their SRI facilitators than Cohort 2 Teams. As was noted earlier. 
Cohort 1 school management teams began their school reform process within a year of the Court's 
decision. Moreover, these teams had minimal time to engage in quality planning. 

Table 5 

Percent of Teams Reporting Satisfaction with their School Review and 
Improvement Teams in Key Areas of Support 



Area of Support 


Percent Reporting Satisfaction 


Assistance with implementation 


56% 


Review of the school's budget 


50% 


Support with resolving problems 


42% 


Assistance with implementation plan 


41% 


Assistance with the development of the school's budget 


39% 


Technical Assistance 


36% 


Attendance at meetings 


29% 


Support with the Model Developers 


26% 


Identifying for the schools, experts who can help with student achievement 


18% 


Identifying areas for training 


15% 


Overall Satisfaction with the SRI Teams 


38% 



Number of teams responding: 37 



Table 6 

ANOVA Results for the Main Effect of Cohort Status 



Dependent Variables 


Df(B)/ 

Df(W) 


MS(B)/ 

MS(W) 


F 


SRI Team 


2 


6.982 


2.996 


attends meetings. 


35 


2.329 




SRI has 


2 


6.775 


3.538* 


helped in 
implementation. 


33 


1.915 




Technical 


2 


8.997 


4.599** 


assistance 
is satisfactory. 


33 


1.956 




SRI has 


2 


7.589 


3.521* 


provided 
assistance with 
implementation plan. 


33 


2.115 




SRI has 


2 


5.699 


2.973 


helped 
in problem 
solving. 


30 


1.917 




SRI has 


2 


6.444 


3.568* 


helped with 
the Model 


32 


1.806 





h A 



Developers, 








SRI has 


2 


3,721 


2,750 


helped to 
identify areas 
for training. 


32 


1,353 




SRI has 


2 


3,643 


2,304 


identified 


32 


1,581 




experts that can 
help with student 
achievement. 








Team is satisfied 


2 


10,836 


5,513** 


with assistance 
from SRI in 

school-based budgeting 


32 


1,966 




Satisfied with 


2 


7,687 


3.757* 


Overall support 
from the SRI 


33 


2,046 





o 

ERLC 



Post Hoc testing based on TUKEY; *p< .05; **p<. 001 

Information provided during the focus group session suggests that two factors were contributing to 
the ineffectiveness of the SRI teams. The first, relates to the instability of team members. All of the 
districts in the focus group concurred that during the early phases of the reform there was a high 
turnover of individuals on the SRI teams. A second contributing factor identified by the districts is 
the knowledge base and experiences brought by the SRI facilitators. There was general agreement 
that the SRI facilitators lacked the experiences and knowledge base around the change process in 
general and reform within the urban context in particular. SRI team facilitators were described as 
being inexperienced and who for the most part seemed to be learning from the districts and schools 
rather than the other way around. These findings on the relative ineffectiveness of the SRI teams 
are not new. An earlier study on factors impacting on the implementation of the reforms pointed to 
problems with the SRI teams and had suggested that the State Department of Education needed to 
closely evaluate the way in which these teams were functioning (Walker & Gutmore, 2000). The 
overall impact of the SRI teams' ineffectiveness is evident in the fact that slightly more that 48% of 
the school based management teams noted that the absence of technical support has posed a 
challenge to their ability to function. 

Understanding the Process of Devolving Authority 

The focus group discussions with central ofOce personnel knowledgeable about the devolving of 
authority to the school management teams as well as discussions with school superintendents provide 
additional insights into the myriad of issues the districts are facing as the shifts in the distribution of 
power and authority occur. All the districts in the focus group prior to the Abbott rulings had begun 
to create opportunities for participatory decision- making in their systems. In some instances, these 
opportunities were more formally structured with the establishment of what is defined as school core 
teams. Thus, districts did not express aversion to devolving authority to the local sites and indeed 
endorsed the process as a means of creating structures that were more inclusive of the voices of their 
various constituents. However, the districts did provide comments on what were perceived to be 
salient issues that adversely affecting the effective implementation of SBM. 

First, there was unanimity among the districts that the vagueness and lack of specificity in the state's 
regulations led to confusion and misinterpretations on the part of the school management teams as to 
their roles and responsibilities. This they pointed out was further exacerbated by the ongoing 
changes to the guidelines that occurred annually. A second related concern dealt with the issues of 
competing power and authority in the areas of school operations and curriculum. Prior to the most 
current form of the regulations there were no statements by the DOE clarifying the overall roles and 
responsibilities of the building principal. This resulted in the school management teams erroneously 
assuming that they were responsible for operational issues within their local schools. A 
compounding factor contributing to the position of the principal vis-a- vis the teams was the 
leadership skills of some principals. District representatives noted that in schools led by weak 
principals, the school management teams emerged as centers of power. Respondents cited examples 
of situations in which these principals had abdicated their responsibilities to the Teams, and in so 
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doing were sometimes unaware of critical decisions made by the teams. 

The importance of properly clarifying the role of the principal in decentralized structures has been 
underscored in some of the literature. According to Meadows (1990) one of the essential problems 
with some forms of school-based management is that the group makes the decision but the leader or 
principal alone is accountable. Research has demonstrated that principal leadership plays an 
important role in the successful devolution of authority. For example, Leithwood et.al (1999) found 
that principal leadership is quite central to teams that have the greatest influence on school 
practices. According to the Leithwood study, the principal's role is both symbolic and 
instrumental. Leithwood noted that school-based management tended to have a greater impact in 
schools in which principals facilitated the development of the teams, helped to focus the teams* 
activities on educationally substantive issues and engaged in a shared or distributive leadership role 
with the teams, than in schools in which the reverse was true. 



The second area of contestation occurred over matters of curriculum. In this arena, central office 
curriculum staff was pitted against the school management teams. According to the regulations, the 
school management teams have considerable responsibilities for ensuring that the curriculum in 
their buildings as well as instruction is aligned to the core content standards. However as the 
districts noted, these curricular issues were previously resolved at the central office level in response 
to the state's adoption of the Core Curriculum Content Standards (which predated the most recent 
Abbott rulings). However, there was uncertainty among the teams about the relationship between 
the enacted curriculum based upon the district's aligned curricular frameworks on the one hand, 
and their responsibility for curriculum in their schools on the other. The confusion experienced by 
the teams with regards to their roles and responsibilities for curricular and instructional matters was 
perceived to be further compounded by the inability of the SRI teams to provide clear directions and 
meaningful guidance to the resolution of these issues. 

As discussed earlier concerns about the effectiveness of the School Review and Improvement Teams 
have been expressed by not only the teams, but central office personnel and superintendents as well. 
Apart from the many issues that were previously mentioned, one extremely problematic area for the 
districts, which surfaced in the interviews with the superintendents, is the SRIs review of transfers. 
The guidelines state that any request for transfers must have the approval of the SRI teams. 
Superintendents complained that this process has not worked efficiently, and that the slow response 
of the SRI teams has created bottlenecks within their organizations. 

Yet in spite of these difficulties, all the districts concurred that their school management teams have 
demonstrated commitment and diligence in their efforts to develop quality implementation plans. 
Most of the districts indicated that their local teachers unions have been instrumental in helping the 
reform process. However, as the districts observed, the rushed timetables for decision-making, the 
inconsistencies and poor guidelines emanating from the DOE and the ineffectual role of the School 
Review and Improvement Teams have all served to undermine the successful devolution of authority 
to the local school sites. 

Discussion and Conclusions 

The findings in this study raise a number of policy concerns regarding how authority gets devolved 
from central to local structures. The first is the apparent tension between policy statements 
developed by state elites and the environments, which they seek to influence. The regulations 
regarding membership composition created two sets of problems for the schools. First, the 
regulations made it clear, that no community member employed by a school district could serve on a 
school management team in the capacity of either a parent or community representative. However, 
in districts, in which the public education sector tends to play a significant role in the employment of 
local residents, this regulation meant that a substantial section of the community would be excluded 
from serving on these teams. Second, the regulations stipulated that no one group of stakeholder 
could constitute a majority on the teams. However, if schools are precluded from recruiting 
memberships from significant pockets within their communities, then the goal of attaining balanced 
representation is difficult to attain. Indeed, the study found, that in effect, among the teams studied, 
there was regression towards a teacher-dominated form of school-based management. The 
preclusion of important “community voices” on these teams resulted in less than favorable 
outcomes. As noted, teams without adequate community representation were less likely to explore 
alternatives before arriving at decisions, and more prone to the intrusion of narrow individual 
interests over group goal. In democratic situations, broad based participation allows for the 
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expression of different viewpoints thus increasing the likelihood of informed decisions being made. 

Policies that do not have as an important corollary, the building of capacity among local actors are 
likely to encounter difficulties during implementation. Moving from centralized to decentralized 
structures imply that at some point during the process, those to whom power is being devolved, will 
develop the necessary prerequisite skills that will allow them to effectively develop and execute 
decisions. The present study found, that teams lacked the experience, knowledge, and skills, and 
were not provided with adequate training that would have allowed them to make effective decisions. 
Furthermore, in the case of the first cohort of schools, the strict timelines imposed by the state on 
these schools to arrive at important decisions regarding their schools instructional programming 
resulted in decisions that were authoritatively rather than democratically made. In some cases, these 
schools* implementation plans were summarily rejected by State elites (Walker, 2001; Walker & 
Gutmore, 2000). 

More importantly, when authority is being shifted or redistributed among various power sites, it is 
important that the spheres of responsibility be thoroughly clarified. In the New Jersey case, no clear 
delineation of the roles and responsibilities of the teams, building principals, central offices, and local 
school boards were made. This led to contestation over areas of responsibilities. In addition to 
clarifying roles, questions as to how the time of teams can be constructively and efficiently used to 
bring about educational improvement in their communities ought to be fully explored. When teams 
lack the capacity to effectively govern, and when there is contestation over spheres of influence 
circumscribing the boundaries of each group's responsibilities is necessary. In the case of the 
current study, it is felt that the roles and responsibilities of the school management teams ought to be 
more circumscribed by state policy. The regulations give the teams a broad set of responsibilities 
that cover most of the processes inhering in teaching and learning as well as the management of their 
schools. However, as was seen, not only do the teams lack the knowledge and experience to fulfill 
some of these tasks, but neither are they comfortable in carrying out some of these functions. 

Further, the rushed timetables for making decisions have made it impossible for the teams to engage 
in quality planning. 

When policies that seek to promote increased school responsibility for decision-making include as a 
precondition the ability of state elites to approve or disapprove decisions that are made by 
democratically constituted teams, these policies in effect undermine the very principles on which the 
concept of decentralization is premised. As was seen in the case of New Jersey, in reality what has 
occurred is an intensification and consolidation of power at the state level. In this case, the school 
systems do not enjoy genuine autonomy, and in reality have only limited discretion over the reforms. 
Thus, decentralization in the Abbott districts has come to function as Ball describes it as a ‘ 
mechanism for delivering reform rather than a vehicle for institutional initiative and 
innovation* (Ball, 1993:76). Clearly, a deconstruction of decentralization within the New Jersey 
context, unmasks the apparent contradictions in the policy governing whole school reform through 
participatory decision- making. This is borne out not only by the data provided in this study, but the 
continuous challenges that have been made to the manner in which the Department of Education has 
reacted to decisions made at the local site (See Walker & Gutmore, 2000). 

Ball (1993) and Smyth (1993) have both suggested that state elites and other interest groups may 
push for decentralization motivated more by protecting their self-interests than any deep-seated 
belief in social democratic principles. In such instances, communities unwittingly grant these groups 
legitimacy. By assuming responsibility for implementing poor policies, parents, teachers and the 
community buffer state elites from any adverse consequences caused by such policies. State elites are 
thus able to avoid their social responsibilities under the guise of decentralization. Moreover, as 
Miron (1996) suggests, one of the unanticipated outcomes of decentralization, is the reinforcement of 
calls by economic elites for market-based solutions to the problem of urban education. Thus, with 
the failure of decentralization the case for privatizing public education can be more forcibly made. 
The issues, which have surfaced in this paper if not addressed substantively at the policy-making 
level, do not augur favorably for empowering local communities to assist in rebuilding their 
educational institutions. 
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Abstract 

Although a great deal has been written about charter schools, rather little attention has been given 
to their funding. The first part of this article raises four current issues in the funding of regular 
public schools across the U.S. and shows how these issues carry over to the funding of charter 
schools. The second part explores four additional issues that have arisen in the funding of charter 
schools that go to the core identity of charter schools and the nature of the students they enroll. 

In both parts, extra attention is paid to developments in California, one of the most active charter 
school states. 



In just over a decade, charter schools have grown from an idea to something of a movement, as more than 2000 
charter schools now serve more than 250,000 in 34 states and the District of Columbia — although this remains but a 
tiny fraction of American school children. Observers disagree whether charter schools have so far turned out to be a 
positive development, although the Consortium for Policy Research in Education (CPRE) has recently released a 
fairly upbeat review of the literature concerning several key policy issues relating to charter schools.* Nevertheless, it 
should be emphasized that this article is not meant as a brief in support of (or against) charter schools. Rather, the 
central point of this article is that if charter schools are to become a regular part of the public school system in at least 
a number of states, then the charter school funding issues discussed here need further attention by both policy analysts 
in the political arena and school finance scholars. It is also perhaps worth emphasizing at the outset that this article 
does not purport to provide solutions to all of the problems examined, although several of California's solutions are 
presented. Instead, it offers something of an agenda of charter school funding topics in need of further policy 
discussion. 

Part I: Four Public School Finance Issues and Their Connection to Charter School Funding 
One: Inter-district inequalities 

A. The issue generally 

The funding of American public schools is historically based on local property taxes. Notwithstanding subventions 
from the state to local school districts, spending per pupil has long varied from district to district.^ More specifically, 
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ever since the 19th century, spending on public education has importantly been a function of the per pupil wealth of 
the local district, with low wealth districts most disadvantaged. 



Because of more than thirty years of litigation that began in the late 1960s, this problem has now been substantially 
ameliorated in some states^— states like Kentucky, New Jersey, Texas, Wyoming and California^— although, even 
today in California, for example, some wealthy communities like Beverly Hills continue to outspend most other 
districts. Overall, however, inter-district spending inequalities remain significant in most states and very large in 
some states.5 (Of course, some people consider wealthier districts entitled to spend more and see "the problem" as 
the interference with that advantage by courts and state legislatures.) 

B. Carry-over to charter school funding 



Inter-district spending inequalities also create a dilemma for charter school funding.^ Either, charter schools 
will be funded (typically by their local sponsoring districts) at a level that relates to the spending level per pupil in the 
districts that charter them— this is the most typical solution around the country.^ Or, they will be funded (perhaps 
directly by the state) at some state average level of funding per pupil— this, for example, is increasingly the California 
solution, 8 Both solutions are problematic. 



If charter funding is tied to district spending per pupil, then charter schools may be very differently funded based on 
who they can get to charter them. This sort of inequality among charter schools surely must seem unfair to many 
charter school operators, and especially so as charter schools begin to lose their connection to families living in a 
particular district and begin to serve children from a metropolitan area. Moreover, this arrangement gives those 
seeking charters special incentives to seek charters from some, but not other, districts. The disincentive applies most 
strongly with respect to low wealth/low spending districts, and these are the very districts that charter school supports 
typically argue have the most to gain from charter schools. 

On the other hand, if charter school funding is provided based on the state average per pupil spending level in public 
schools, then this discourages the conversion of existing public schools to charter schools in high spending districts, 
and it also makes it hard for new charter schools to compete in districts that have high spending. State average 
spending also artificially encourages conversions to charter schools in low spending districts. At the same time, 
regular public schools in those low spending districts would understandably feel unfairly disadvantaged as compared 
with charter schools with which they compete. 

Because California has substantially eliminated inter-district inequalities in the core funding for public education, 
these inter-district problems for charter schools are far less there than in most other states. What this also shows is 
that if more states were to reduce their inter-district inequalities, this would arguably be good both for public schools 
generally and for charter school funding. Indeed, it is plausible that the reform of the funding of regular public 
schools could actually be effectuated through the spread of charter schools because of the tensions and inconsistencies 
that are so clearly exposed by the finance of charter schools. 



Two: Intra-district inequalities 

A. The issue generally 

In many school districts, teachers (and other employee positions) are funded in this way. The school district awards 
each local school one teacher slot for every X number of pupils it has. Then, whatever that teacher's salary, it is fully 
paid for centrally.^ This means that schools with higher paid (in general, more experienced) teachers get more money 
spent per pupil on their core teaching staff than do those schools with lower paid teachers. 



Because of teacher seniority rights and other factors, this often means that schools serving the lowest income pupils 
have the lowest spending per pupil on their core faculty. *0 This is vividly apparent in a community such as Oakland, 
California, where the higher achieving schools in the Oakland hills generally have much better paid and more 
experienced teachers, and the lower achieving schools serving low income children who live in the Oakland flats tend 
to have large concentrations of lower paid teachers (many of whom are new to the job and working with only 
temporary teaching credentials). 



This issue of intra-district inequalities arising from the way teacher slots are funded was supposed to have been 
changed in Los Angeles in response to litigation. And indeed, it appears that the Los Angeles Unified School District 
has now infused schools that previously spent less than the district average on core teacher salaries with substantial 
new money, although those schools have not generally been able to attract their share of more senior teachers and 
have instead used the funds for other core instructional purposes. ' • Despite this improvement in intra-district equity, 
this litigation does not appear to have been copied elsewhere, and, in any event, intra-district inequalities are 
widespread throughout the nation. (Again, some people don't see this as a "problem," because they favor higher per 
pupil funding on core teachers in neighborhoods where wealthier families live, perhaps to offset the funding 
advantages that other neighborhoods have because of categorical programs aimed at children from low-income 
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households.) 



B. Carry-over to charter school funding 



Charter schools, as already noted, are generally funded on a per pupil basis, at least for the core of their resources. 
When this method is used, it eliminates the issue of intra-district spending inequalities just described — at least among 
charter schools. But it also creates some thorny incentives for existing public schools. For example, per pupil 
funding at the school level will probably make it financially quite unwise for local public schools to convert to charter 
schools if their teachers currently are more experienced and higher paid. By contrast, schools with low teacher 
salaries on average have a special, perhaps artificial, incentive to convert. 

Funding charter schools on a per pupil basis but regular public schools on a teacher position basis also means that, to 
survive financially, charter schools will probably have to rely upon a large cadre of mainly newer and lower paid 
teachers as compared with the more attractive schools in the district that chartered them. This understandably seems 
unfair to the charter schools. 

It should be clear, then, that were regular public schools (like charter schools) financed on a per pupil basis for their 
core funding, those schools with concentrations of higher paid teachers would probably have to make do with fewer 
of them (which would imply a larger average class size than today for regular public schools stocked with senior 
teachers). But at the same time, regular public schools with lower paid teachers would probably be able to hire more 
teachers and reduce class size. Such a change would both be better for the neediest schools and would mesh regular 
school funding with charter school funding. Hence, just as noted earlier with respect to inter-district spending 
inequalities, it is also plausible that financial the inconsistencies laid bare by the spread of charter schools will 
actually stimulate the reform of public school funding within districts. 

Three: Inadequate spending 

A. The issue generally 

Despite its wealth and previously high level of spending on public education, California currently spends much less 
per pupil than do comparably wealthy states, and overall spends less than many school finance experts believe is 
necessary to facilitate high quality outcomes for most students. Although state-to- state comparisons are difficult 

(because of different accounting measures and different costs from place to place), roughly speaking California 
schools today tend to spend around $6000 per pupil annually for current expenditure items, whereas the figures in 
places like New Jersey and Connecticut are nearly twice that number. Some blame this low level of spending on 
public schools in California on the adoption in the 1970s of Proposition 13, which both sharply reduced local property 
tax collections and restricted future increases; ^ at least one scholar has blamed this decline in relative spending in 
California directly on the school finance litigation that reduced inter-district inequalities; If^ still another scholar 
blames California’s relatively low spending on the unwillingness of state politicians to invest in education as 
substantial a share of the personal incomes of the people of California as are political leaders in many other states. 



Although California, given the overall wealth of its population, may be seen by some as especially miserly towards its 
schools, it is hardly at the bottom in terms of public school spending. Indeed, there are many much poorer states 
around the country — like Louisiana, Alabama and Mississippi — that also spend quite modestly on their public school 
pupils, even taking their relatively lower labor and other costs into account. (Once more, of course, some believe 
that too much money is already spent on public education in America and would see "the problem” more in terms of 
"excessive" spending in states like Connecticut and New Jersey.) 

5. Carry-over to charter school funding 



Given that public school budgets are seen as tight in many states, this means that many states and local school 
districts do not feel themselves able to provide generous funding of charter schools. This shows up in a variety of 
forms. 



1. Start-up costs are often lacking or inadequate. These are the funds needed to launch the school, to hire 
staff, and to outfit the school with furnishings and curriculum materials in preparation for its initial 
enrollment of what is often an uncertain number of pupils. In some communities this issue has been 
reasonably well addressed because charter schools are able to tap into special funding available from the 
federal government, special state programs, and private foundation grants.-^ Nevertheless, start-up costs 
often remain a serious problem, especially for newly formed community groups that seek to create charter 
schools. 

2. In some communities, the fees charged for "oversight" and for "services" provided to charter schools by 
their sponsoring districts have been set at quite high levels, and some charter school supporters charge that 
this is likely to be especially true in districts that are not all that keen to sponsor charter schools in the first 



place 2 1 Although this issue remains quite troubling in some states, it has been at least partly addressed in 
recent California statutory provisions which now both cap the oversight fee and clearly give charter schools 
the ability to go to outside providers for the business and other services they need.22 
3. Apart from schools that convert from existing public schools, charter schools often obtain little or no extra 
money to pay for their school buildings (whether those buildings are leased or bought and paid for with 
mortgage funding) 23 when this happens, it means that charter schools often have to redirect perhaps 20% or 
more of their core funding to pay for space, leaving them with what they believe to be too little to pay for the 
ongoing educational program. 2^ Indeed, an inadequate supply of school facilities may be the single largest 
stumbling block to the growth of charter schools.25 

In California, however, the recently adopted Proposition 39 promises to resolve this issue starting in 2003. 
This initiative proposition insists that bodies approving charter schools provide those schools either with 
adequate school buildings or the money to rent or buy them.2b Yet, it remains to be seen how charter school 
rights under this new law will play out. Indeed, this new requirement might cause many districts to decide to 
call a halt to chartering, even though California law is supposed to make it difficult for districts to refuse a 
charter sought by any applicant with reasonably sensible plans for its school.27 in any event, this problem 
remains a difficult one for charter schools in other states 



4. Despite the points just noted, it should also be appreciated that sometimes charter schools have certain 

financial advantages. For example, in states that provide uniform per pupil funding for charter schools at all 
grade levels, charter schools have tended to be created to serve elementary, not high school, pupils because 
younger children are traditionally cheaper to educate.28 (California’s program resists this trend by offering 
charter schools more money per pupil for education at the higher grades.29) a second advantage in some 
places is that the calculation of the per pupil allocation to charter schools includes sums that regular public 
schools spend on services, like transportation, that charter schools might not provide. Both of these points 
illustrate the importance of determining the proper spending benchmark to which charter school funding is to 
be tied. 

Yet, as a general matter, until regular public schools are reasonably well funded, it is probably unrealistic to 
expect generous funding for charter schools (although some people don’t consider this to be a "problem," 
believing that a system of charter schools is only worth the governmental effort if those schools are as good 
or better than regular public schools at a much lower cost to the taxpayers.) 



Four: Special needs funding 

A. The issue generally 

Federal funding programs for educationally disadvantaged children (through the federal Title I program) and for 
children with disabilities (through the federal special education law— IDEA) implicitly assume that, if left to their 
own discretion, all too many local public schools, districts and states would choose to treat these groups of children 
badly. This explains why Congress decided that civil rights sorts of protections are necessary for these special needs 
children. One consequence, however, is that the federal restrictions attached to the distribution of these ’’categorical’’ 
funds burden local schools in various ways. 

First, in order to make sure that schools actually spend federal funds on educationally disadvantaged children, federal 
regulations impose substantial reporting and accounting burdens on schools. These, in turn, have caused some 
schools to use the money in ways other than they believe would be best for those children (although recent federal 
changes have sought to reduce this problem).30 Second, even though the federal government has recognized the 
special needs of disabled children. Congress has failed even to provide adequate extra funds to deal with those extra 
needs. 21 Yet, Congress has mandated that those needs be met. As a result, there is considerable pressure on local 
school districts to shift money otherwise earmarked for the rest of their children to these especially needy pupils. This 
shift is called encroachment" by experts and is now thought to be quite substantial in the special education area, with 
some observers claiming that as much as 25% of the funds intended to regular education must be siphoned off to pay 
for the extras needed by special education pupils— that is, required extras that are not paid for by federal and state 
categorical special education funds. 22 Most people would surely agree that it is good that these needy children get 
extra money, but it must be appreciated that, as a result, even less money is left for other children, and that is 
generating some backlash among parents of non-disabled pupils. 



B. Carry-over to charter school funding 

Many charter schools have had great difficulty getting their legally entitled share of state and federal categorical funds 
from the appropriate state or local funding authority through which federal and state categorical funds flow. 

In California now, this issue may have been fairly well resolved with respect to most of the smaller state-funded 
categorical aid programs. This is because charter schools are supposed to obtain their share of those funds on a per 
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capita basis — i.e., they receive so many extra dollars for every pupil in the charter school without having to show that 
they actually have enrolled pupils eligible for those extra categorical funds, without having to fill out all the otherwise 
required forms, and without having to show how they have spent those extra funds. This is very desirable for 
charter schools, both because the cash is much more likely to be received in a timely matter and because the charter 
school can avoid the red tape tied up in claiming and receiving funds from a large number of small categorical 
programs. 

But the receipt by charter schools of their share of federal Title I funds and funds for special education pupils remain 
substantial issues. One key factor to appreciate here is that some charter schools either avoid enrolling, or don't really 
know much about teaching, special education pupils with anything more than very modest disabilities. Those schools 
are viewed in some quarters as shirking their fair share of these pupils. At the other extreme, it must also be 
understood that other charter schools deliberately specialize in, or simply wind up as magnets for, children with 
disabilities.^^ The upshot is that there tends to be a very uneven distribution of special education children in 
individual charter schools as compared with the state average.-^ 

As a result, especially for high cost special education children, it is essential that extra funding is provided to charter 
schools on an individual pupil basis. That is, simply increasing the charter school's per capita allocation based on the 
total number of children in the school — as has been done in California for small categorical programs — would clearly 
give too little money to some charter schools and too much to others. Moreover, many advocates on behalf of 
children with special needs remain skeptical about the good faith of many charter schools in enrolling and fairly 
treating such children. Hence, they are no more willing to surrender regulatory controls over categorical funding for 
charter schools than they are for regular public schools, even though such children only enroll in charter schools "by 
choice."^^ 

But actually arranging for the receipt by charter schools of extra funding for those needy pupils has been difficult in 
many places and has tended to immerse charter schools in bureaucratic controls from which they are, as a general 
matter, supposed to be free. Moreover, charter schools are new in many places and the necessary bureaucratic 
linkages are just getting formed. Furthermore, charter school operators are often unsophisticated in timely completing 
forms and carrying out procedural activities that it has taken local public schools years to master. In addition, one 
suspects that some local districts are not all that eager to spread categorical funds to charter schools (just as they have 
been slow to pay over federal categorical funds to eligible private schools), especially at a time when, as noted, they 
receive inadequate amounts of these funds to serve the pupils enrolled in their own regular public schools. Indeed, the 
encroachment problem has caused some districts to want to cut the basic funds they provide to charter schools (before 
adding back categorical funds) on the ground that the district's own regular public schools are suffering from this 
problem. Needless to say, many charter school operators find this cutback in the funds they think they should receive 
hard to accept, or even understand. Until these matters are better resolved, some charter schools will view themselves 
as mistreated whereas others will continue to shun the neediest children. 

Part II. Four Special Issues of Charter School Funding 

One: How to Count Charter School Pupils 

As already explained, chartering bodies generally fund charter schools on a per pupil basis. This makes it essential 
for there to be a fair, and fairly run, system for counting how many pupils the charter school actually serves. But 
several issues have arisen, both because of fraud committed by some charter schools and, arguably, unduly harsh 
counting mechanisms imposed by some chartering bodies. 

Conceptually, the first decision is whether to count pupils on the basis of ADA (average daily attendance), or 
ADM/A DE (average daily membership or enrollment), or yet some other pupil-counting measure.-^ This, of course, 
is an issue for regular public schools as well as for charter schools. The difference is important. For example, a 
school will have more pupils enrolled than attend on any given day, and hence ADM/ADE counts, other things equal, 
will bring in more money. But in addition, notice that if ADM/ADE is used, schools with high absentee rates, in 
effect, get extra funds that can be used to help them track down pupils who are not attending regularly and to support 
efforts designed to coax those pupils back into school full time. 

For charter schools, an ADM/ADE pupil count may create an incentive both to manufacture enrollment numbers on 
the days ADM/ADE is counted and to treat as enrolled, but then later ignore, regular truants. On the other hand, if 
ADA is used, then charter schools have an incentive to discourage the enrollment in the first place of those students 
who are likely to have high absentee rates.^^ This problem of perverse incentives is likely to be especially 
troublesome for charter schools, as compared with regular public schools, because the former inevitably will have 
more control over who their pupils are. 

One possible solution to this problem is to use an ADM/ADE count in the funding of charter schools, but then to 
make sure that all enrollees are counted when assessing student achievement at the school, thereby making it 
undesirable for charter schools to ignore enrollees who become truant. The bottom line is that, absent a well -crafted 
solution to this issue, the charter school funding mechanism can wind up having an unintentionally large impact on 
which pupils charter schools seek to attract. 
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Two: Schools that seek to enroll distance learners/home schoolers 



Nationwide it is now estimated that perhaps as many as 2% of children are home schooled by their parents.^9 should 
those parents who home school their children be able to group together as a charter school in order to obtain financial 
support from the government? This is not really an issue for regular public schools, because these children 
conventionally are simply not enrolled in "school" (although some enterprising public school officials, as a way of 
gamering more state funding for the district, have figured out how to enroll in their "independent study" programs 
some children who are, in effect, home schooled). 

If home schoolers are permitted to join charter schools, this will increase the cost of public education. This new cost 
alone will make some state officials want to resist funding charter schools established to serve these sorts of children. 

Nonetheless, in some states, including California, home-schooled children may be served by charter schools. This has 
created several problems, however. Part of the problem stems from the fact that many have come to the conclusion 
that these children can be served at a public cost level much below that of other children and, in turn, below the per 
pupil allocation normally made for children enrolled in charter schools (assuming that the teacher/parents will not be 
paid for their work). 

Consider this example. Suppose that the state is prepared to provide $5,000 annually for each pupil enrolled in a 
charter school, whether home schooled or not. But suppose that reasonably good educational support for the home 
schooling family can be provided at a cost of, say, $3000 per pupil per year. That leaves $2000 left over, and some 
have seized upon this difference, concluding that there is money to be made. That "profit" might be skimmed off by 
the chartering school district, at least if it can get its hands on the $5000 per pupil from the state and then provide only 
$3000 per pupil to the charter school serving home schoolers. Alternatively, the $2000 profit might be skimmed off 
by the charter school operator (which itself might be a "for-profit" educational management organization). In this 
scenario, the charter school gets the full $5000, but only spends $3000 on the pupil. 

Beyond the matter of educating home schoolers "on the cheap," it has turned out, in California at least, that some 
districts have chartered schools to serve home schoolers who live at a great distance from the charter school, and 
furthermore, in several instances, the distance-leaming-based charter school's center of operations itself has been 
located far from the sponsoring district.'^^ This scenario causes at least two worries. One is that pupils will be 
claimed as enrolled in a charter school when they in fact are not in any way connected with the charter school. In 
California recently, a variation on this theme apparently happened, when the same children were claimed as enrollees 
of two different charter schools, both of whom claimed to manage their education from afar.^^ A second problem is 
that the sponsoring district's oversight of the charter school may be minimal. . 

These various concerns, fueled in part by scandals linked to home-schooling-based charter schools, have caused some 
in California to want to eliminate the participation of home schoolers in charter schools and others to favor providing 
a lower per pupil payment to charter schools serving such pupils.^2 yet, the argument on the other side is that 
these children, like all other children, are supposed to be entitled to free public education. Moreover, many home- 
schooled children seem to be learning reasonably well, and hence it seems unfair to some to target them for worse 
treatment, especially when lots of those children who are home-schooled might actually benefit substantially from 
public spending on their behalf. This spending could, for example, be for curriculum development and curriculum 
materials, training and supervision of their parents, computer technology, and the like. Yet, even some strong charter 
school supporters who abstractly identify with families who home-school their children are conflicted, fearing that 
scandals involving these schools put the charter school movement at risk. As a result, the place of these children in 
the charter school system remains a difficult issue. 

Three: Monitoring charter school spending, budgeting, auditing, etc. 

The basic idea behind charter schools is that they are supposed to be autonomous. That is, it is broadly agreed that the 
core principle defining charter schools is that they are generally to be free from regulation in order to be able to 
experiment, to be flexible in the way they manage their operations, to respond quickly to their customers, and so 
on.^3 In return for this autonomy, charter schools usually are asked to demonstrate academic outcome results for their 
children, but that too is supposed to be measured without too much interference with the school's independence."^^ 

And yet, there is at the same time understandable concern that some charter schools will be financial rip-offs. They 
might not properly spend on their pupils the public money they get; they may go broke in the middle of the term and 
leave children and families in the lurch; and so on. These fears have led to auditing, monitoring, reporting, and other 
requirements. 

Principals and teachers in regular public schools, of course, may also complain about these same controls. But, in 
contrast to charter schools, freedom from regulation is not traditionally so centrally part of their school's identity 
(although school site control, of course, has long been a goal of some reformers of regular public schools). In any 
event, some charter school advocates believe that excessive financial controls are becoming the back door way that 
the charter school movement is undermined. Indeed, the history of public school funding may provide a revealing 
lesson here, since increased state funding has generally brought with it more regulation of local school district 




operations. 



Four: Supplemental funding of charter schools 

Some charter schools are required to live off the basic funding they get from their chartering body. As noted already, 
many have to devote too much of that to pay for their building. Other charter schools, as noted, have the advantage of 
operating in a building that has been given to them, or loaned to them, for free. This creates "haves" and "have-nots" 
among charter schools. But this general problem of financially better and worse off charter schools is not limited to 
buildings. 

For both start-up costs (mentioned earlier) and for ongoing operations, some charter schools have substantial 
supplemental funding and others do not. To the extent that this extra funding comes from government grants, one can 
at least hope that the funds will be fairly awarded to the most deserving charter schools. But much of this 
supplemental funding comes from private donors. Perhaps one should not begrudge a charter school that is able to 
obtain this extra funding. And the availability of this supplemental funding may provide charter schools with a 
healthy incentive to convince private donors that they have a high quality program deserving of their support. 

Yet, the uneven distribution of outside funding creates inequalities of the sort that are also created among regular 
public schools when, for example, some obtain extra funding from local privately-financed public school foundations 
and others do not. At least for regular public schools, however, those foundations currently make a relatively small 
difference for a relatively few school districts. 

But for charter schools, there is a much greater concern that no charter school will have much of a chance to succeed 
unless it has substantial extra outside funding, at least in its early years. This, in turn, means that certain sort of 
charter school initiators are far more likely to survive than others, with local grass roots groups most likely to be in 
the worse off category. For many who envisioned that charter schools would be a kind of democratic, local 
community response to regular public schools, this is likely to be disheartening. 

Conclusion 

Several of the funding issues facing charter schools stem from broader issues underlying the funding of public schools 
generally. Oddly enough, if the charter school system grows and legislatures struggle to rationalize the charter school 
funding mechanism, there is at some chance that these changes will, in turn, force attention to, and changes in, the 
funding of regular public schools. Other charter school funding issues that go to the heart of what autonomy charter 
schools will truly have and what sorts of pupils charter schools will serve. This overview shows that charter school 
funding is far too important a topic to leave to technical experts and deserves the careful attention of the wider school 
reform community. 
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Premack, supra note 8, at 21-22. 



“^Cal. Educ. Code section 47613. 



23premack, supra note 8, at 33. See generally United States General Accounting Office, Charter Schools: Limited 
Access to Facility Financing (September 2000)(describing the inadequacy of funding available for charter school 
facility financing). 



24premack, supra note 8, at 69, 77. 

2^Paul T. Hill The Supply Side of School Choice in School Choice and Social Controversy, supra note 6, at 140-73. 



^^Premack, supra note 8, at 33. 

^^Sugarman, supra note 20, at 893-907. 

^^Nelson, supra note 1, at 36. 

Premack, supra note at 27. 

^^Stephen D. Sugaiman, Two School-Finance Roles for the Federal Government: Promoting Equity and Choice, 1 7 
St. Louis U. Public L. Rev. 79, 80-81 (1997). 
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'^^Senate Bill 740, 2001-2002 Reg. Sess. (Cal. 2001)(Oct. 14, 2001 Version), available at 
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Aprender a Ensenar Para La Sociedad del Conocimiento 

Carlos Marcelo 
Universidad de Sevilla 



Resumen 

Nuestras sociedades estan envueltas en un complicado proceso de transformacion. Una 
transformacion no planificada que esta afectando a la forma como nos organizamos, como 
trabajamos, como nos relacionamos, y como aprendemos. Estos cambios tienen un reflejo visible 
en la escuela como institucion encargada de formar a los nuevos ciudadanos. Nuestros alumnos 
disponen hoy en dia de muchas mds fuentes de informacion que lo que ocum'a no hace ni diez 
anos. Fuentes de informacion que, aportadas por las nuevas tecnologias de la informacion y 
comunicacion, estan haciendo necesario un replanteo de las funciones que tradicionalmente se 
han venido asignando a las escuelas y a los profesionales que en ella trabajan: los profesores y 
profesoras.^En que afectan estos cambios a los profesores? ^Como debemos repensar el trabajo 
del profesor en estas nuevas circunstancias? ^Como deberian formarse los nuevos profesores? 
^Como adecuamos los conocimientos y las actitudes del profesorado para dar respuesta y 
aprovechar las nuevas oportunidades que la sociedad de la informacion nos ofrece? iQue nuevos 
escenarios educativos y escoiares son posibles/deseables? En este articulo hacemos una revision 
de los recientes avances en la investigacion sobre aprender a ensenar y sobre los procesos de 
formacion. 

Abstract 

Modern societies are engulfed in a complicated process of transformation. This unplanned 
transformation is affecting the way we are organized, how we work, how we relate to each other, 
and how we learn. These changes are visible in the institutions charged with educating new 
citizens. How are these changes influencing the role of teachers? How do we begin to rethink 
teachers’ work in these new circumstances? How should the new generation of teachers be 
developed? What knowledges and attitudes must we instill in today's teachers so that they may 
take advantage of the new opportunities offered by the information society? In this article we 
review the literature of teacher learning and teacher knowledge from the point of view of new 
challenges presented by the knowledge society. 



1. Introduccion 

Nuestras sociedades estan envueltas en un complicado proceso de transformacion. Una transformacion no planificada 
que esta afectando a la forma como nos organizamos, como trabajamos, como nos relacionamos, y como aprendemos. 
Estos cambios tienen un reflejo visible en la escuela como institucion encargada de formar a los nuevos ciudadanos. 
Nuestros alumnos disponen hoy en dia de muchas mas fuentes de informacion que lo que ocurria no hace ni diez anos. 
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Fuentes de informacion que, aportadas por las nuevas tecnologias de la informacion y comunicacion, estan haciendo 
necesario un replanteo de las funciones que tradicionalmente se han venido asignando a las escuelas y a los 
profesionales que en el la trabajan: los profesores y profesoras. 

Una de las caracteristicas de la sociedad en la que vivimos tiene que ver con el hecho de que el conocimiento es uno de 
los principales valores de sus ciudadanos. El valor de las sociedades actuales esta directamente relacionado con el nivel 
de formacion de sus ciudadanos, y de la capacidad de innovacion y emprendimiento que estos posean. Pero los 
conocimientos, en nuestros dias, tienen fecha de caducidad y ello nos obliga ahora m^s que nunca a establecer 
garantias formales e informales para que los ciudadanos y profesionales actualicen constantemente su competencia. 
Memos entrado en una sociedad que exige de los profesionales una permanente actividad de formacion y aprendizaje. 



^En que afectan estos cambios a los profesores? ^Como debemos repensar el trabajo del profesor en estas nuevas 
circunstancias? ^Como deberian formarse los nuevos profesores? ^Como adecuamos los conocimientos y las actitudes 
del profesorado para dar respuesta y aprovechar las nuevas oportunidades que la sociedad de la informacion nos 
ofrece? ^Que nuevos escenarios educativos y escolares son posibles/deseables? 



Las preguntas anteriores configuran todo un programa de preocupaciones que esta llevando a muchos academicos, 
profesionales, investigadores, padres y docentes, etc. a pensar en que la escuela tiene que dar respuesta pronta a los 
desafios que se le avecinan. Respuestas que van directamente relacionadas con la capacidad de ofrecer la mejor 
educacion a la que todos los alumnos tienen derecho. Y para ello volvemos la vista hacia el profesorado que trabaja 
codo a codo con nuestros estudiantes. ^C6mo se forman? ^que conocimiento realmente necesitan? ^que cambios hay 
que introducir en su formacion para que sean de nuevo los lideres de un cambio que la sociedad esta demandando? 
^como aprenden los profesores? ^que nuevas estrategias y compromisos son necesarios? ^como se plantea una 
profesion docente en una sociedad del conocimiento donde cualquiera puede acceder a la informacion y -quizas- 
convertirse en ensenante? 



En este articulo vamos a repasar que sabemos sobre los procesos de formacion de los profesionales de la ensenanza, asi 
como sobre la ensenanza como profesion. Y plantearemos que se pueden observar elementos de la actual formacion 
inicial de los profesores que no responden ni por extension ni por estructura a los desafios que anteriormente 
enunciabamos. Una formacion inicial que es insuficiente en muchos paises -por ejemplo Espana- en el caso del 
profesorado de educacion infantil y primaria, y que es marcadamente obsoleta en lo que respecta al profesorado de 
educacion secundaria, incluyendo bachillerato y ciclos formativos. A diferencia de la formacion inicial de los 
profesores, la formacion continua ha evolucionado a traves de diferentes modelos de formacion que poco a poco han 
mostrado sus aciertos y debilidades. 



Los cambios en la forma de aprender, que afectan a los profesores en ejercicio, estan acentuando la idea de que la 
responsabilidad de la formacidn recae cada vez mas en los propios profesionales. Hacer de nuestras escuelas espacios 
en los que nos solo se ensena sino en los que los profesores aprenden, representa el gran giro que necesitamos. Y para 
ello, nada mejor que entender que es el derecho a aprender de nuestros alumnos, el principio que debe orientar la 
formacion. Una formacidn dirigida a asegurar un aprendizaje de calidad en nuestros alumnos, comprometida con la 
innovacion y la actualizacion. Que supere el tradicional aislamiento que caracteriza a la profesion docente. Una 
formacion que consolide un tejido profesional a traves del uso de las redes de profesores y escuelas y que facilite el 
aprendizaje flexible e informal. Una formacion en definitiva que contribuya a reprofesionalizar la docencia frente a 
aquellos que pretenden simplificar la complejidad del acto de ensenar. 

2. Algunos Cambios Que Nos Estan Afectando 

Los ultimos anos del siglo pasado y los que llevamos de este nos vienen mostrando que la humanidad ha entrado en 
una nueva era que afecta a la produccidn, la energia, las comunicaciones, el comercio, el transporte, el trabajo, la 
formacion o la familia. Estos cambios ya los podemos observar con mucha nitidez, puesto que estan afectando 
progresivamente nuestra forma de vivir, de comunicamos, de trabajar, y de aprender. El declinar de la familia nuclear, 
ha promovido la aparicion de una amplia variedad de modelos familiares, entre las que figuran las familias 
monoparentales (entre un 10-15% en Europa segiin Vonk, 1998), las familias en las que se agrupan los hijos de 
diferentes matrimonios (a mitad de los 90 el promedio de divorcios en los paises de la OCDE fue del 40%), y las 
familias en las que los dos miembros trabajan. La creciente -y positiva- incorporacion de la mujer al mercado de 
trabajo esta creando espacios de desatencion hacia los hijos, no por culpa de las mujeres, sino por la no existencia de 
politicas pr^cticas de apoyo para la atencion de los ninos y jovenes en periodo de tiempo fuera de la escuela. Y lo 
mismo podriamos decir de la atencion a las personas mayores. 



Los cambios en las condiciones familiares estan afectando y mucho a la escuela y a su funcion social. Hay coincidencia 
en admitir que la escuela hoy dia -utilizando la escuela en terminos generales para referirme a las instituciones 
educativas de educacion obligatoria- esta asumiendo funciones de socializacion, de cuidado y atencion, que antes 
correspondian a la familia (Senge, 2000). Pero tambien vemos una escuela que empieza a sentir el descenso de la 
natalidad (en la mayoria de los paises de la OCDE ha caido por debajo de 2), asi como se constata un incremento 
visible del envejecimiento de la poblacion. Por primera vez, en los paises de la OCDE la proporcion de personas 
mayores de 65 anos se equipara a los menores de 15 (OCDE, 2001). 
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Y nos encontramos con unas nuevas generaciones -denominada generacion Nintendo- con una mayor competencia en 
el uso de las nuevas tecnologias, con una activa presencia en el mercado como consumidores activos, pero, segun el 
informe de la OCDE titulado Schooling for tomorrow, con una cierta tendencia al individualismo, fragmentacion y poca 
implicacion poHtica. 

Los Cambios en las personas y en las instituciones tienen que ver con otros cambios mas profundos que se estan 
produciendo en la manera de entender los sistemas social, economicos y politicos. Una de las caracteristicas del mundo 
en el que vivimos y en mayor medida en el que viviran las generaciones que hoy dia estan en la escuela, es la creciente 
diversidad cultural, religiosa, etnica. El aumento constante de la inmigracion en los paises de la OCDE, motivada por el 
aumento de las diferencias entre paises ricos y pobres, esta poniendo a nuestras sociedades ante diatribas nunca antes 
contempladas. Sociedades acostumbradas a la monoculturalidad y al monolingiiismo, ven como sus escuelas se Henan 
de chicos y chicas de procedencias geograficas, culturales y lingiiisticas muy diferentes. Y el discurso de la 
interculturalidad se enfrenta al de la multiculturalidad; el de la integracion al de la segregacion; el de la seguridad al de 



Pero los cambios a los que anteriormente me he referido no se explican sin una mirada a las profundas transformaciones 
que en las ultimas decadas se han producida en los sistemas economicos y productivos. Existe actualmente un discurso 
plenamente legitimado que destaca el papel de la economia en la definicion de los problemas de las personas y las 
sociedades. Una economia que evoluciona, cambia sus patrones, sus identidades y que se hace mds difusa y global. Una 
economia que permite que, como si de un efecto mariposa se tratase, una crisis economica en los paises asiaticos 
produzca una recesion en Chile. Una economia que destaca de manera creciente la importancia de la Gestion del 
Conocimiento en las empresas, pero que se permite jubilar anticipadamente a trabajadores experimentados con mas de 
50 ahos, o contratar a trabajadores procedentes de empresas de trabajo temporal. 

Una de las caracteristicas de la nueva economia es su estructura en red y su dependencia del conocimiento y de las 
nuevas tecnologias. Se trata de una economia en la que la productividad y el poder estan mas asociadas a la generacion, 
procesamiento y transmision de la informacion que a la transformacion de las materias primas (Castells, 1997). Una 
sociedad informacional cuyas caracteristicas son segun este eminente sociologo espahol: 

• La informacion es su materia prima, son tecnologias para actuar sobre la informacion, no solo informacion para 
actuar sobre las tecnologias como en el caso de revoluciones previas. 

• La capacidad de penetracion de los efectos de las nuevas tecnologias. Puesto que la informacion es una parte 
integral de toda actividad humana, todos los procesos de nuestra existencia individual y colectiva estan 
directamente moldeados por el nuevo medio tecnologico. 

• La logica de la interconexion de todo sistema o conjunto de relaciones que utilizan estas nuevas tecnologias. La 
configuracion de red se acomoda a las nuevas relaciones. 

• Flexibilidad: los procesos se flexibilizan y se acomodan; flexibilidad de trabajo de tiempo de distancias. 

• Convergencia creciente de tecnologias especificas en un sistema altamente integrado: la microelectronica, 
telecomunicaciones, optoelectronica, ordenadores se integran en sistemas de informacion. 

El discurso sobre la nueva economia nos habla de un horizonte laboral diferente al que actualmente conocemos. Castell 
afirma que "en general, la forma tradicional de trabajo, basada en un empleo de tiempo completo, tareas 
ocupacionales bien definidasy un modelo de carrera profesional a lo largo del ciclo vital, se estd erosionando de 
manera lenta pero segura” (\991\191), Y parece que la palabra flexibilidad resume la nueva manera de entender el 
contrato social. Flexibilidad de tiempos, de espacios, de conocimientos, de tareas, de relaciones, de trabajo, etc. 
Flexibilidad entendida como un valor que el trabajador debe poseer para poder seguir sobreviviendo. Flexibilidad para 
acomodarse a nuevas situaciones y contextos laborales, para desplazarse s61o o con su familia a otro pais o continente; 
flexibilidad para aprender nuevas habilidades en el lugar donde se encuentre, en el trabajo, en casa o en un hotel; 
flexibilidad para coordinarse con trabajadores de otras partes del mundo que participan en el proceso de produccion 
dentro de su propia empresa. 

Flexibilidad sostenible es el titulo de un informe que la OCDE publico en 1997. En el se afirma que "una nueva 
organizacion social y economica basada en las tecnologias de la informacion intenta descent ralizar la gestion, 
individual izar el trabajo, y adecuar el mercado a los consumidores, a la vez que fragmenta el trabajo y segmenta a las 
sociedades. Al mismo tiempo las nuevas tecnologias facilitan la descentralizacion de las tareas laborales y su 
coordinacidn mediante una red de comunicacion en tiempo real, ya sea entre continentes o entre diferentes pisos del 
mismo edificio. " (p.9).Las empresas de la nueva economia tienden a una produccion flexible y adaptada a la demanda, 
personal izando los productos en funcion de las necesidades del cliente. Esa produccion flexible, que intenta mejorar la 
produccion en serie propia de la era industrial, tiene unas repercusiones importantes en los procesos laborales y en la 
Concepcion del puesto de trabajo. La fragmentacion de la produccion esta llevando a que el proceso de creacion de un 
producto puede realizarse en lugares muy distantes. Piezas de decoracion que se disenan en Espana y que se fabrican en 
China. Componentes informaticos que se fabrican en Taiwan y se integran en Irlanda. 
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Otra de las caractensticas de la nueva economia, a la que anteriormente aludimos es su estructura en red: "las redes son 
el elemento fundamental del que estdn y estardn hechas las nuevas organ izaciones"(CsiStc]]s, 1997: 196). Las 
caractensticas de esta estructura han contribuido a una produccion flexible, a la interconexidn entre las empresas, a la 
existencia de una estructura empresarial mas horizontal, organizada en tomo al proceso, no a la tarea, con jerarquias 
planas, gestidn en equipo, medida de los resultados por la satisfaccidn del cliente; recompensas basadas en los 
resultados del equipo; maximizacidn de los contactos con los proveedores y los clientes; infoimacion, retencion y 
foimacion de los empleados en todos los niveles (Castells, 2001). Estas caractensticas destacadas tambien por el 
informe de la OCDE al que hemos hecho referencia anteriormente: "Los dos elementos claves de esta transformacion 
son la flexibilidad de los procesos del trabajo y el funcionamiento en red de las empresas, tanto internamente como en 
sus relaciones con el exterior. Por flexibilidad entendemos la adaptacion constante de las tareas a realizar sobre 
productos, procesos y mercados que cambian, asi como la creciente autonomia de los trabajadores a los que se les 
demandan mayores habilidades, capacidad de autogestion, y responsabilidad individual. Por funcionamiento en red 
(networking) nos refer imos a una nueva logica de las empresas, en las que las jerarquias y formas organizativas estdn 
basadas en conexiones interactivas entre diferentes capas y posiciones dentro de la empresa, entre empresas y con el 
mercado. Las nuevas tecnologias de la informacion permiten una mayor flexibilidad y funcionamiento en red; y la 
globalizacion destaca la interdependencia, la interaccion y la constante adaptacion a un ambiente cambiante" (OCDE, 
1997:12). 

Una Clara repercusion de los cambios que estamos destacando tiene que ver con el concepto de trabajo. La 
incorporacion de las nuevas tecnologias de la informacion esta redefiniendo los procesos laborales y a los trabajadores 
y, por tanto, el empleo y la estructura ocupacional. Por una parte, se esta demando una mejor foimacion para una 
cantidad considerable de puestos de trabajo mientras que otra gran cantidad esta desapareciendo por la automatizacion 
tanto en la fabricacion como en los servicios. Los trabajos a tiempo parcial, temporales, el horario flexible de trabajo y 
el autoempleo estan creciendo en todas las sociedades. Una de las repercusiones de este fenomeno es el aumento 
creciente de los denominados teletrabaj adores. Son profesionales, cuyo puesto de trabajo es volante. Su oficina puede 
ser su casa, un telecentro o simplemente un ordenador portatil conectado a internet desde el que se envian datos a la 
oficina central. Padilla comenta en relacion con el teletrabajo, que "es una formula organizativa compleja (permite 
organizar a los empleados de forma que se rompe el binomio espacio-tiempo u oficina tradicional-horario de trabajo, 
permitiendo el que estos puedan realizar sus funciones de manera mas flexible), que viene a cambiar estructuras 
establecidas desde hace decadas, no por arbitrariedad de sus defensores sino por razones economicas 
(fundamentalmente reduccion de costesy aumento de la productividad) y organizativas (fundamentalmente 
adaptabilidad y flexibilidad), que conviene conocer, ya que permite que la empresa pueda obtener una ventaja 
competitiva sostenible basada en compete nc ias y capacidades" (PadiWa, 1998, XIII). 

El teletrabajo es una respuesta de las empresas, pero tambien de los trabajadores individuales (autoempleados) para 
aprovechar las oportunidades que las nuevas tecnologias ofrecen en provecho de una mayor rentabilidad y 
competitividad. Oportunidades que se generan en la medida en que los trabajadores (por cuenta propia o ajena) estan 
conectados y aprovechan las ventajas de Internet. Oportunidades que pueden hacer que mas personas puedan 
incorporarse al trabajo independientemente de sus condiciones fisicas, como es el caso de los minusvalidos (Fernandez 
Villalta, 1998). 



El teletrabajo, en sus distintas formas coincide en una creciente disgregacion e individualizacion de los procesos y de 
los propios trabajadores. Elio no quiere decir necesariamente aislamiento, ya que a la vez que aumenta la 
especializacion, se generaliza lo que se ha llamado "toyotismo", o forma de organizacion empresarial que achata la 
tradicional estructura piramidal de las organ izaciones empresariales, tendiendo hacia una estructura mas horizontal en 
la que el trabajo en equipo, la coordinacion y colaboracion se priman como formas de asegurar la calidad de los 
procesos de produccion. 



Esto que hemos enunciado tiene repercusiones en lo que se ha venido entendiendo como el "desempeno del puesto de 
trabajo". La hiperespecializacion y rutinizacion de tareas ha dado paso a una necesaria multifuncionalidad de tareas a 
realizar por los trabajadores. Por ello es frecuente el desplazamiento de funciones y tareas dependiendo de las 
necesidades asi como de las innovaciones introducidas en el proceso de produccidn o venta. Todo ello repercute en que 
los trabajadores vean continuamente incrementadas y diversificadas las tareas que deben realizar, lo que lleva a tener 
que aprender continuamente nuevas habilidades y conocimientos. Esto tendra una repercusion muy importante en la 
foimacion continua como veremos mas adelante. 

Las innovaciones tanto en los procesos gestion, de produccidn, de control de calidad, como de ventas y 
comercializacion generan nuevos conocimientos y habilidades que los trabajadores deben incorporar continuamente. El 
mercado de trabajo es cambiante y a la vez que elimina ocupaciones obsoletas, abre nuevas vias de desarrollo y 
crecimiento. El informe Oferta y demanda de empleo cualificado en la nueva economia muestra que las empresas de 
informatica y telecomunicaciones son las que mas empleo generan en Espana: El 14.09% de las ofertas de trabajo 
proceden del sector informatico, mientras que el 9.84% del de las telecomunicaciones, lo que supone un 23.93% de los 
empleos (El Pais, 1 3 de agosto, 2000). 

Pero no solo las ocupaciones relacionadas con las nuevas tecnologias son las que ofrecen nuevos espacios laborales. Se 
estan produciendo en nuestra sociedad cambios que conducen necesariamente a la aparicion de nuevas ocupaciones, o 
lo que se ha venido en llamar nuevos yacimientos de empleo, definidos como "aquellas actividades destinadas a 
satisfacer nuevas necesidades sociales que actualmente se configuran como mercados incompletos, que sean intensivas 
en el empleo y que tengan un dmbito de produccion/prestacion definido en el territorio" (Jimenez, Barreiro y Sanchez, 
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cit. Por Garcia Fraile, 2000, p. 284). El Informe encargado por el Ministerio de Trabajo y Asuntos Sociales, y 
elaborado por Cachon (1998) destaca los siguientes factores en la aparicion de estos yacimientos de empleo: 

• La inversion de las piramides de poblacion, consecuencia del alargamiento de la esperanza de vida y de la 
caida de natal idad. 

• La extension de los sistemas educativos, lo que determina nuevas demandas educativas y foimativas en el 
sistema escolar y en el de foimacion peimanente 

• La importante incorporacion de la mujer al mercado de trabajo y la transfoimacion de las estructuras 
familiares. 

• El desarrollo de las nuevas tecnologias de la infoimacion y la comunicacion. 

• Los cambios en las pautas de consumo y del uso del tiempo social, de ocio y cultura. 

Estos factores estan creando posibles espacios en los que es posible desarrollar iniciativas de empleo. Se ban agrupado 
en los siguientes: 

• Servicios de la vida cotidiana: servicios a domicilio, atencion a la infancia, nuevas tecnologias de la 
infoimacion y la comunicacion, ayuda a los jovenes con dificultad. 

• Servicios de mejora de la calidad de vida: mejora de alojamientos, seguridad, transportes colectivos locales, 
revalorizacion de espacios urbanos, comercio de proximidad, gestion de la energia 

• Servicios de ocio: turismo, audiovisual, valorizacion del patrimonio cultural, desarrollo cultural local, deporte 

• Servicios medioambientales: gestion de residuos, gestion del agua, proteccion y mantenimiento de zonas 
rurales, reglamentacion y control de la polucion y las instalaciones correspondientes 

Estos yacimientos de empleo no representan por si solos sino oportunidades, opciones que pueden fructificar si se 
dispone de apoyos, iniciativa y conocimiento. Tambien estan influidos por factores sociales y culturales que pueden 
favorecerlos o dificultarlos. Y uno de los factores determinantes es poder disponer de personas con conocimientos y 
formacion para hacer realidad las demandas de la sociedad. Por ejemplo, en relacion con las nuevas tecnologias de la 
infoimacion se viene insistiendo en la necesidad de disponer de personas cualificadas para desempenar ocupaciones 
relacionadas con ordenadores. Se ha sabido la oferta que Alemania ha hecho para que especialistas hindues se 
desplacen a Alemania atrabajar en ocupaciones relacionadas con las nuevas tecnologias. Igualmente se esta hablando 
de la necesidad de "reconvertir" a licenciados en Fisica, Quimica, Matematicas, y otros hacia estudios informaticos. 



Todo lo dicho anteriormente lleva a reconocer el hecho — ya sabido- de que en la sociedad que nos esta tocando vivir, 
los conocimientos que adquirimos en nuestra etapa de formacion inicial tienen wm/echa de caducidad. No podemos 
seguir esperando que la formacion profesional inicial nos dote de un bagaje de conocimientos del que podamos 
disponer a lo largo de toda nuestra vida profesional activa. Por el contrario, tanto por la aparicion constante de nuevas 
ocupaciones y profesiones, como por el imparable avance de los conocimientos, se requiere de las personas, de los 
ciudadanos una actitud de peimanente aprendizaje. 

Las situaciones que anteriormente he descrito nos muestran un panorama ya vislumbrado cuando la Union Europea 
encargo en 1996 el informe titulado Teaching and Learning. Towards the Learning Society. Lo que se ponia de 
manifiesto en aquel informe era que los cambios que se estaban produciendo generaban unas necesidades de formacion 
y unas actitudes en los ciudadanos que debian atenderse para poder aprovechar con toda su amplitud las oportunidades 
que la sociedad informacional iba a generar. Y se apuntaba hacia el peligro de exclusion social que una falta de 
respuesta adecuada podria producir, afirmando que existe el riesgo de una grieta en la sociedad entre aquel los que 
pueden interpretar; aquel los que solo pueden usar; y aquellos que quedan fuera de la sociedad y dependen de ella para 
sobie vivir: en otras palabras, entre los que conoccn y los que no conocen. 
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Tanto por las condiciones de trabajo como por las oportunidades de empleo que anteriormente hemos comentado, se 
percibe la necesidad de contar con una ciudadania con una foimacion de base lo suficientemente fundamentada como 
para que pueda darse esa flexibilidad y adaptacion a la que nos hemos referido. Y esta formacion de base es la que debe 
proporcionar el sistema educativo obligatorio. Una formacion que, frente a la temprana especializacion que algunos 
plantean, debe atender a aspectos de formacion general. A este respecto, el informe de la propia Comision Europea 
hablaba de que la escuela deberia desarrollar un conocimiento base que permita a las personas dar significado a las 
cosas, comprender y hacer juicios, desarrollar la capacidad de analizar como funcionan las cosas: observacion, sentido 
comun, curiosidad, interes por el mundo fisico y social, inventar. Aprender a cooperar debe ser tambien una habilidad a 
adquirir en la escuela puesto que las modernas empresas estan organizdndose en circulos de calidad, que introducen la 
planificacion entre trabajadores, destacando su autonomia. Un conocimiento que permita a los ciudadanos dominar los 
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idiomas que le faciliten una mayor movilidad laboral y tambien cultural. 

Adalberto Ferrandez insistid a lo largo de su vida en la importancia de entender la formacion de base como el sustrato 
imprescindible para cualquier propuesta foimativa con opciones de exito. El hablaba de que tendria que ser una 
educacion con moldes de 'algebra conceptual', es decir, creadora de estructuras flexibles, mentalmente hablando, a la 
vez que polivalente: estructuras abiertas que puedan ser modiflcadas rapidamente, de acuerdo a los datos que surgen de 
su intomo y entomo (1988: 174). La formacion de base debe aspirar a permanecer en un mundo donde todo cambia, a 
dotar a las personas de autonomia personal, capacidad de comunicacion, conocimiento de los procesos de resol ucion de 
problemas, manejo de informacion, etc. 



Plantear la formacidn de base como un caleidoscopio mas que como un binocular, conduce a asumir que su funcion no 
es mas que permitir que las personas puedan continuar pedaleando aunque cambie el terreno o la forma de la bicicleta. 
Lo que aprendemos en un momento de nuestra vida tiene una utilidad relativa en funcion de los avances del 
conocimiento producido por la investigacion. 

Como consecuencia de ello, poco a poco nos vamos dando cuenta que la division clasica entre el mundo del estudio y el 
mundo del trabajo esta dejando de tener sentido. La idea de que existe un tiempo para la formacion (basica, inicial) en 
la que adquirimos el bagaje de conocimientos que vamos a necesitar para toda nuestra vida profesional no se mantiene 
hoy en dia. La formacion inicial es una formacion basica que nos permite empezar a desenvolvemos en el mercado 
laboral. Pero el mercado laboral es todo menos estable. Muchas profesiones u ocupaciones surgiran en los proximos 
aftos que aun hoy en dia no sospechamos de su existencia. Por otra parte, el incremento exponencial de los 
conocimiento hace que lo que aprendemos en la formacion inicial tenga una fecha de caducidad fijada. Como decia 
Delors en su informe, es que ya no basta con que cada individuo acumule al comienzo de su vida una reserva de 
conocimientos a la que podra recurrir despues sin limites. Sobre todo, debe estar en condiciones de aprovechar y utilizar 
durante toda la vida cada oportunidad que se le presente de actual izar, profundizar y enriquecer ese primer saber y de 
adaptarse a un mundo en permanente cambio. 

La necesidad de aprender a lo largo de toda la vida se ha convertido en un lema cotidiano. Zabalza (2000) hablaba de 
que hemos convertido "/a agradable experiencia de aprender algo nuevo cada dia, en un inexcusable principio de 
supervivencia** {\6S). Y en nuestro contexto, las estructuras y procesos que facilitan ese aprendizaje toman el nombre 
de formacidn. En otro tiempo uno se formaba para toda una vida, hoy dia nos pasamos la vida formandonos. Y la 
formacion se nos aparece como el dispositivo que empleamos para adaptar la formacion de base que hemos adquirido 
(educacion secundaria, universitaria, profesional, etc.) a nuestras necesidades o las de la empresa en la que trabajemos. 

3. El Aprendizaje en el Punto de Mira: El Derecho de Aprender y el Papel de las Escuelas 

Las transformaciones que nuestras sociedades estan viviendo no pasan por delante de las escuelas sin llamar a la puerta. 
Quizas sea el mundo educativo en general, pero el escolar en particular, el que menos se haya dado como aludido por 
las profundas transformaciones que se estan produciendo y a las que anteriormente me he referido de manera somera. 
Una pleyade de pensadores e investigadores estan mirando desde un punto de vista critico a nuestros actuales sistemas 
escolares para someterlos a consideracion. El informe de la OCDE al que me referia anteriormente concluia con una 
declaracion que me parece de interes reproducir: '^Quizas la transformacion mas urgente y dificil pero necesaria que 
deben fiacer los paises de la OCDE sea organizar la escuela en torno a una educacion post-secundaria universal, que 
promueva la independencia, ajustes rdpidos al cambio y movilidad. Los sistemas educativos en el pasado se han 
utilizado no solo para impartir habilidades cognitivas sino como selectores sociales, distribuyendo a los ninos de 
diferentes procedencia socioeconomicas en sus niveles ”apropiados" de educacion, que les permitiera elegir trabajos 
apropiados. Esto ha funcionado razonablemente bien en un sistema industrial jerdrquico basado en trabajos poco 
cualificados y con pocos cambios en la vida laboral de los individuos... Hoy en dia, esa estratificacion resulta 
socialmente mucho mas contraproducente que en el pasado. Los jovenes con educacion secundaria se encuentran cada 
vez mas en situacion de riesgo en gran medida porque tanto el sistema educativo como los empleadores los ven como 
sujetos poco preparados para trabajos de gran habilidad y flexible'' (2001 : 36-37). 

Y no es que la principal meta de la escuela sea la de preparar para el trabajo. Es que una ciudadania activa no puede 
construirse con sistemas educativos obsoletos en cuanto a su organizacion y estructura tanto didactica como curricular. 
Unos sistemas escolares que siguen anclados en los principios de seleccion y clasificacion, donde se asume que los 
alumnos llegan a las escuelas con deficiencias que las escuelas deben arreglar; que el aprendizaje tiene lugar en la 
cabeza y no en el cuerpo en su conjunto; que todos aprenden o deberian aprender de la misma forma; que el aprendizaje 
tiene lugar en las aulas, no en el mundo; que hay chicos listos y torpes y que eso es inevitable; que el conocimiento es 
por naturaleza fragmentado, que la escuela comunica la verdad, y que el aprendizaje es principalmente individualista y 
la competicion acelera el aprendizaje (Senge, 2000). Tanto el curriculo como la forma de organizacion del trabajo en el 
aula que actualmente se lleva a cabo en nuestras escuelas no se adecua a las necesidades de educacion de la nueva 
ciudadania. Como planteaban Osin y Lesgold "Las escuelas convencionales agrupan a los estudiantes de la misma 
edad en periodos fljos de tiempo. No existen razones educativas que puedan justiflcar este enfoque. La diversidad de 
ritmos de aprendizaje de los individuos muestra que es absurdo esperar que todos los alumnos en una misma cohorte 
de edad aprendan la misma cantidad de contenidos en la misma cantidad de tiempo" (1996:644/ 



Chapman y Aspin (2001), editores del International Handbook of Lifelong Learnings plantean la necesidad de realizar 
profundas transformaciones en los sistemas educativos actuales para que podamos enfrentamos a los desafios de la 
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sociedad del conocimiento. Estos autores plantean una serie de principios que nos parecen de interes: 



• La necesidad de ofrecer oportunidades educativas que respondan a los principios de: eficacia economica, 
justicia social, inclusion social, participacion democratica y desarrollo personal. 

• La necesidad de reevaluar los curriculum tradicionales y las formas de ensenar en respuesta a los desafios 
educativos producidos por los cambios economicos y sociales y las tendencias asociadas al surgimiento de una 
economia del conocimiento y una sociedad del aprendizaje. 

• La reevaluacion y redefinicion de los lugares donde el aprendizaje tiene lugar, asi como la creacion de 
ambientes de aprendizaje flexibles que sean positivos, estimulantes y motivadores, y que superen las 
limitaciones de curriculo estandarizados, division por materias, limitados tiempos y rigidas pedagogias. 



• Una aceptacion de la importancia del valor anadido que aporta el aprendizaje. 

• La consciencia de que aunque se empiece a entender que la escuela no sea la principal fuente de adquisicion de 
conocimiento, se esta convirtiendo en institucion fundamental en la socializacion de la gente joven. 

• La idea de que los itinerarios de aprendizaje entre las escuelas e instituciones de educacion superior, 
trabajadores y otros proveedores de educacion tendra un alto impacto en la formacion de relaciones entre la 
escuela y la comunidad. 

• La necesidad de promover la idea de la escuela como comunidad de aprendizaje y como centros de aprendizaje 
a o largo de la vida. 

Son muchos los autores que vienen decantandose en la misma direccion: la necesidad de una profunda y fundamentada 
reflexion acerca de las nuevas misiones de las escuelas como instituciones que promueven el conocimiento y el 
aprendizaje a lo largo de la vida. Keating (1998) plantea la necesidad de que las escuelas se conviertan en 
organizaciones que aprenden. Darling-Hammond (2000) insiste en la necesidad de que las escuelas garanticen el 
derecho de aprender de los alumnos. Dalin (1998) habla claramente de que las escuelas actuales en absoluto estan 
preparadas para ayudar a los alumnos a enfrentarse con las realidades del siglo XXL Se requieren escuelas y profesores 
comprometidos con el aprendizaje continuo, flexible, en colaboracion. Escuelas que promuevan una ensenanza para la 
comprension, la diversidad, la indagacion (Dalin y Rust, 1996). 

Pero la evolucion de los sistemas educativos no parece hoy en dia clara. Aunque las senales nos indican que hay una 
creciente necesidad en colocar el aprendizaje en el centro del debate, poco se ha ido ava:nzando hasta ahora. Y el riesgo 
que empezamos a correr es que ya va viendose que aprendizaje no es sinonimo de escuela (o de Universidad). Que las 
escuelas puede que no sean las unicas instituciones educativas. Y que, por tanto, la profesion docente acabe 
desprofesionalizandose. Dalin y Rust (1996) se planteaban algunas preguntas y daban algunas respuestas que nos 
parecen de mucho interes para el planteamiento que estamos haciendo. Se preguntaban, ^En el siglo XXI? 

iQuey Quien Serdn Los Alumnos?: En el futuro los alumnos seran desde los recien nacidos hasta los adultos y 
mayores. El mensaje aqui es que lo importante es el que aprende y la escuela debe dar respuestas a las necesidades. 



/iQue Y Quien Sera Profesor?: El profesor tradicional seguira siendo importante, pero la distincion entre profesor y 
estudiante sera mas diluida. Los estudiantes podran ser profesores y los profesores deberan seguir aprendiendo. Los 
padres podran ser recursos adecuados. Habra contribuciones de voluntarios. 

iQue Curriculum?: Sera muy dificil definir un curriculum comprehensivo, debido a la revolucion del conocimiento y 
debido a las necesidades cambiantes de los alumnos. 

^Que Libro De Texto?: La informacion proviene del libro, del ordenador, de CD-ROM. 



iQue Sera Una Clase? La mayor parte dc la informacion no esta en la escuela, sino en ordenadores, bibliotecas, locales 
comunitarios, en los medios, en los lugares de trabajo. 

En la misma linea, pero planteando diferentes opciones, se ha mostrado la OCDE, a traves de su informe Schooling for 
tomorrow. What School for Future?, al que ya hemos aludido. En este in forme se plantean diferente escenarios de 
desarrollo que pueden seguir los sistemas educativos y que evidentemente repercuten tanto en la calidad de ensenanza, 
en el trabajo y consideracion de los profesores, como en la organizacidn de la educacion. 
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Desescolarizaci6n 
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1 . Sistema escolar burocrdtico 


3. Escuelas como centros sociales 


5. Redes de aprendizaje y sociedad 
en red 


2. Extension del modelo de 


4. Escuelas como organizaciones de 


6. Exodo de profesores 


mercado 


aprendizaje 



En la pagina siguiente reproducimos una sintesis de esos escenarios que pueden ayudamos a comprender las 
posibilidades, pero tambien los riesgos de no afrontar los cambios necesarios en la direccion adecuada. Los seis 
escenarios responden a tres marcos generales. El primero tiene que ver con el mantenimiento de la situacidn actual de 
las escuelas con un funcionamiento burocratico y curricular centralizado, como servicio publico, impartido por un 
profesorado funcionario, y con una clara intencion de clasificacion. A este modelo se le pueden anadir liberalizaciones 
que permitan que las empresas puedan acceder al mercado de la educacion, bien a traves del establecimientos de 
centros privados o bien mediante esponsorizacion. Al respecto afirmaba Robertson (1998) que "Ya sea mediante 
cuasiprivatizacion o por privatizacion completa, las escuelas son tan atractivas que las grandes empresas estdn 
preparadas para potenciar cualquier movimiento que les permita un mayor acceso a las escuelas y al enorme 
potencial de mercado que los ninos representan. En este clima de creciente recorte de recursos para las escuelas, los 
profesores han empezado a buscar patrocinios con empresas y recursos curriculares para aumentar las posibilidades. 
Cuando uno estd hambriento, incluso una Big Mac parece buena" (4 1 2). 



Un siguiente escenario se plantea como consecuencia de una profunda reestructuracion de las escuelas como 
organizaciones que aprenden y que son capaces de generar espacios de indagacion y cooperacion con padres y 
sociedad. Con amplio apoyo y financiacion publica y con un fuerte compromiso de los profesores con el desarrollo, 
mejora y calidad de la educacidn que imparten. Escuelas caracterizadas por promover un aprendizaje continuo, por 
tener claridad de sus metas, por poseer estandares academicos altos y universales, un curriculum interdisciplinar y 
multicultural, comprometida con la idea del estudiante como trabajador y como ciudadano, con una evaluacion basada 
en el rendimiento en competencias claramente definidas, un clima respetuoso, la implicacion de las familias, toma de 
decisiones compartidas, compromiso con la diversidad de los estudiantes y del profesorado, eleccion libre del centro 
por los estudiantes y tiempo de planificacion conjunta para el profesorado (Darling-Hammond, 2001). 



Los dos ultimos escenarios dibujados por la OCDE corresponden a situaciones en las que las escuelas dejan de ser 
instituciones educativas hegemonicas y aparecen nuevas oportunidades de aprendizaje que utilizan internet como 
vehiculo de adquisicidn de formacion y conocimiento. Esta es una realidad presente ya en algunos paises, como en 
Estados Unidos de Norteamerica en donde se estima que 1 ,5 millones de ninos han dejado de ir a las escuelas para ser 
educados por sus padres en casa (Senge, 2000). Esto se ex plica tambien por el teletrabajo al que nos referimos al 
comienzo de este texto. 
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permanente y 



Privatizacidn, 
acuerdos publico/ 
privado, sistema de 
cheque. 



Individualizacion y 
educacidn en casa 
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Existen Redes donde 
hay ganancias 



Reduccidn 
sustancial de las 
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de educacidn. 

Agencias 
internacionales de 
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Un profesorado 
indiferenciado, un 
nuevo 

profesionalismo con 
diferentes perfiles: 
publico, privado, 
tiempo 

completo ,parcial. 

Profesionales a la 
demanda, 
dependientes del 
mercado. 



Aparicidn de 
acreditacidn para 
profesionales que 



largo de la vida es 
explicito. 



Reduccion de 
desigualdades y 
ampliacidn de la 
diversidad 



certificacidn. 

Aprendizaje a lo 
largo de la vida 
es explicito. 

Inversidn en 

reducir 

desigualdades 



Escuelas menos 
burocrdticas, mas 
diversas. 

Escuela abierta, 
estudiantes 
diferentes, mayor 
mezcla 

intergeneracional. 

Revisidn de las 
divisiones entre 
educacidn primaria y 
secundaria 

Mayor uso de TIC en 
especial para 
comunicacidn entre 
alumnos, padres, 
profesores, 
comunidad 



Nuevas formas de 
organizacidn. 

Dimensidn local de 
la escuela apoyada 
en casos de 
infraestructuras 
ddbiles. 



Cuerpo de profesores 
con alto estatus pero 
no a tiempo 
completo. 



Tipos de contratos 
variados 



Otros profesionales, 
padres, etc. 

M^ relacidn entre la 
enseflanza con otras 
responsabilidades en 
la comunidad. 



Escuelas como 
organizaciones 
que aprenden. 
Organizaciones 
mds pi an as. 
Enfoque de 
equipos. 



Atencidn al 
nuevo 

conocimiento 

Amplia mezcla y 
variedad en edad, 
grados y 
habilidad 
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desarrollada 

como 

herramienta de 
aprendizaje, 
andlisis y 
comunicacidn. 
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las escuelas, 
educacidn 
terciaria e 
industrias del 
conocimiento 



Aprendizaje 
individualizado o a 
traves de redes de 
alumnos, padres y 
profesionales. 
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utilizadas para 
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Profesorado con 
alto estatus, 
disfrutando de 
buenas 

recompensas y 
condiciones. 

Variedad de 
contratos 

Mayor 

innovacidn en la 
enseflanza y 
aprendizaje. 

Estar en redes es 
la norma entre 



Medios de 
comunicacidn 
juegan fuerte para 
desmantelar el 
sistema escolar. 

Nuevas formas de 
acreditacidn 
intemacional para 
las elites. 

Las autoridades 
publicas 
encargadas de 
cerrar la "brecha 
digital" 



Las redes aportan 
formacion 
dependiendo de las 
necesidades. 
Aparecen nuevos 
profesionales del 
aprendizaje. 
Aparece la ayuda 
en linea y la visita 
a domicilio. 



privadas 



Di versa 
respuesta 
organizativa a la 
falta de 
profesores. 
Vuelta a 
metodos 
tradicionales en 
respuesta a caida 
de estandards 



Amplio uso de 
TIC como 
altemativa a 
escasez de 
profesores 



Se incrementa el 
interns de las 
empresas y 
medios de 
comunicacidn en 
la educacidn. 

"Prdstamos" de 
profesores entre 
paises 



Incremento de 
incentivos para 
profesores 

Peores 

condiciones de 
trabajo con 
problemas en 
peores dreas 

Esfuerzo por 
incorporar a 
profesores 
jubilados 

Crecen semi- 
profesionales 
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Estos ultimos escenarios contemplan tambien la extension de la educacion mas alia de las fronteras de las escuelas y en 
algun caso vienen a reconocer, al igual que hizo Ivan Illich que la educacion debe ser una tarea visible socialmente y que 
debe implicar a todas las instituciones publicas y privadas. Es la idea de las ciudades educadoras que asumen un 
compromiso claro y mantenido de las autoridades publicas, empresas privadas, instituciones educativas e investigadoras, 
organizaciones de voluntarios, con el objetivo de mantener el aprendizaje como centro del desarrollo de la ciudad. Junto a 
ello, se plantea una clara estrategia que ofrezca oportunidades de aprendizaje a todas las edades, creando servicios basados 
en el conocimiento; mejora de las capacidades individuales y organ izativas para crear un ambiente de aprendizaje, 
innovacion, creatividad y cambio, asi como una atencion a la cohesion social y ambiental como parte integral del 
desarrollo de la ciudad (Chapman y Aspin, 2001). 

4. Y Las Miradas Se Vuelven Hacia Los Profesores 

Los Cambios y las nuevas realidades que hemos desglosado brevemente en las paginas anteriores nos dirigen 
inexorablemente la mirada hacia los docentes. Day afirmaba que *7o5 profesores son potencialmente el activo mas 
importante de la vision de una sociedad del aprendizaje*' (2001:495). Y efectivamente es asi. Si queremos que los peores 
escenarios descritos mas arriba no sucedan, debemos prestar atencion muy concreta a los profesores, a su formacion 
inicial, a su periodo de insercion profesional y a su formacion continua. El aprendizaje a lo largo de la vida no es solo un 
buen slogan. Representa una necesidad urgente para revitalizar una profesion demasiado castigada y desmovilizada desde 
un punto de vista profesional. Cranston (1998) se preguntaba si los profesores estan preparados para ensenar en el nuevo 
milenio. Y responde que si miramos el trabajo de los alumnos en sus clases y colegios, lo que encontramos es mas un 
modelo del siglo XIX. Afirma que se demanda un cambio en la forma como se concibe la profesion docente: Un 
profesional que no dedicara toda su vida profesional en un unico sistema educativo o incluso en un unico pais, un 
trabajador del conocimiento. Y, de la misma manera que los estudiantes, los profesores deben prepararse para trabajar en 
un ambiente cambiante e impredecible, en donde el conocimiento se construye desde diferentes fuentes y perspectivas. 

Pero ocurre que a la tarea de ensenar los profesores siguen enfrentandose generalmente en solitario. Solo los alumnos son 
testigos de la actuacion profesional de los profesores. Pocas profesiones se caracterizan por una mayor soledad y 
aislamiento. A diferencia de otras profesiones u oficios, la ensenanza es una actividad que se desarrolla en solitario. Como 
de forma acertada afirma Bullough, la clase es el santuario de los profesores.. .El santuario de la clase es un elemento 
central de la cultura de la ensenanza, que se preserva y protege mediante el aislamiento, y que padres, directores y otros 
profesores dudan en violar (1998). Cuando estamos asistiendo a propuestas que evidentemente plantean la necesidad de 
que los profesores colaboren, trabajen conjuntamente, etc., nos encontramos con la pertinaz realidad de profesores que se 
refugian en la soledad de sus clases. Ya resulta clasico el estudio llevado a cabo por Lortie en 1975, en el que mediante 
entrevistas establecio algunas caracteristicas de la profesion docente en Estados Unidos, que no solo son de gran 
actualidad, si no que son perfectamente aplicables a nuestro pais. Una caracteristica identificada por Lortie fue el 
Individualismo. Este individualismo se produce en opinion del autor por la ausencia de ocasiones en las que los profesores 
puedan observarse unos a otros, y ello se produce desde los primeros anos de formacion como profesor y posteriormente 
durante el proceso de socializacion. 



El aislamiento de los profesores esta favorecido evidentemente por la arquitectura escolar, que organiza la escuelas en 
modulos estandar, asi como por la distribucion del tiempo y el espacio, y la existencia de normas de independencia y 
privacidad entre los profesores. El aislamiento, como norma y cultura profesional tiene ciertas ventajas y algunos evidentes 
inconvenientes para los profesores. En este sentido senalan Bird y Little (1986) que aunque el aislamiento facilita la 
creatividad individual y libera a los profesores de algunas de las dificultades asociadas con el trabajo compartido, tambien 
les priva de la estimulacion del trabajo por los companeros, y se deja de recibir el apoyo necesario para progresar a lo largo 
de la Carrera. Es mas, cuando en el mundo de las empresas se esta hablando de la necesidad de gestionar el conocimiento 
como medio para rentabilizar ese saber hacer que los empleados han ido acumulando a lo largo del tiempo, en la 
ensenanza, en palabras de D. Hargreaves, los profesores "ignoran el conocimiento que existe entre ellos; por tanto, no 
pueden compartir y construir sobre este conocimiento. Al mismo tiempo tampoco conocen el conocimiento que no poseen 
y por tanto no pueden generar nuevo conocimiento. Hay una compleja distribucion social del conocimiento en la escuela: 
ningun profesor en particular conoce o puede conocer la totalidad del conocimiento profesional que los profesores 
poseen" {\999\\1A). Ello es debido a que gran parte del conocimiento de los profesores es tacito, dificil de articular y el 
objetivo de la gestion del conocimiento consiste en ayudar a la organizacion a utilizar su propio capital intelectual. 

El aislamiento representa un barrera real frente a las posibilidades de formacion y de mejora. Los cambios que se estan 
produciendo en la sociedad inciden en la demanda de una redefmicion del trabajo del profesor y seguramente de la 
profesion docente, de su formacion y de su desarrollo profesional. Los roles que tradicionalmente han asumido los 
docentes ensenando de manera conservadora un curriculum caracterizado por contenidos academicos hoy en dia resultan a 
todas luces inadecuados. A los alumnos les llega la informacion por multiples vias: la television, radio, ordenadores, 
Internet, recursos culturales de las ciudades, etc. Y los profesores no pueden hacer como si nada de esto fuera con ellos. 
Salomon nos ofrecia su metafora respecto a que se esta modificando el rol del profesor desde transmisor de informacion, el 
solista de una flauta al frente de una audiencia poco respetuosa, al de un disenador, un guia turistico, un director de 
orquesta (1992:42). Asi, el papel del profesor deberia de cambiar desde una autoridad que distribuye conocimientos hacia 
un sujeto que crea y orquesta ambientes de aprendizaje complejos, implicando a los alumnos en actividades apropiadas, de 
manera que los alumnos puedan construir su propia comprension del material a estudiar, trabajando con los alumnos como 
companeros en el proceso de aprendizaje. 




Los cambios en los profesores no pueden hacerse al margen de como se comprende el proceso de aprendizaje de los 
propios profesores. ^como se aprende a ensenar? ^como se genera, transforma y transmite el conocimiento en la profesion 
docente? Unos cambios que se concretan en formas distinta de entender el aprendizaje, la ensenanza, las tareas, asi como 
los medios y la evaluacion (Blumenfeld, 1998), 
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Unos cambios que deben llevar a replantear el trabajo de los profesores en el aula y la escuela, que conduzcan a una 
estructura escolar mas flexible y adaptada a las posibilidades y necesidades individuales de los alumnos. For ello se 
requiere un replanteamiento en la educacion primaria y secundaria, tanto de los contenidos como de la forma de 
ensenarlos. Y en cuanto a los contenidos academicos existe la tendencia a incrementar las materias, los programas, 
ampliando el numero de boras de dedicacion a las disciplinas escolares. Y si lo que se persigue no es el almacenamiento de 
la informacion y la repeticion rutinaria de tareas, sino la comprension de lo que se aprende, habria que asumir el principio 
enunciado por Gardner y Boix: menos es mas. Con ello vienen a confirmar que el principal enemigo de la comprension es 
completar el temario, la compulsion de tocar todo el libro de texto, en lugar de dar tiempo para presentar material es desde 
multiples perspectivas (1994:203). Pero empenarse en que los alumnos comprendan en lugar de que meramente recuerden 
no resulta gratuito para los profesores. Requiere aprender la forma de implicar a los alumnos para que construyan el 
conocimiento de una forma mas activa, participando y colaborando con companeros, requiere un conocimiento mas 
profundo de la materia que se ensena, asi como de la forma de representarla para hacerla comprensible a los ninos. 



Comprender lo que se aprende y aprender a aprender configuran dos demandas imprescindibles para la escuela actual. Y 
ello es aplicable tanto a los profesores como a los alumnos. Hemos comentado anteriormente que las caracteristicas de la 
sociedad actual en relacion a la mundializacion de la economia esta ejerciendo una gran influencia en las formas de trabajo 
y en las habilidades y actitudes que las empresas demandan de los trabajadores. La capacidad y la disposicion para 
sobrevivir en todos los sentidos del termino estan ahora mas que nunca asociados a la capacidad de aprender y a la 
motivacion por aprender. Y en esto las escuelas primarias y sobre todo las secundarias estan fallando. Por supuesto que los 
factores asociados a la pobreza influyen en estas altas tasas de abandono, pero quizas tambien un curriculum esclerotizado 
y poco adaptado a las necesidades y posibilidades de los alumnos. 

La escuela debe promover en los alumnos una identidad propia como personas con capacidad de aprender, de ser 
responsables y de emprender. Una capacidad para aprender que se concreta en lo que se ha denominado aprendizaje 
autorregulado, mediante el cual se genera en los alumnos un estilo propio de implicarse en la resolucion de tareas, 
estableciendo sus propias metas, planteando sus propias estrategias para evaluar el grado de cumplimiento de las metas, 
procesando informacion y encontrando recursos para aprender. Y esto que es necesario para los alumnos en una sociedad 
cambiante lo es tambien para los profesores. Mas adelante profundizaremos en este tema pero por ahora planteamos la 
necesidad del aprendizaje autonomo para los profesores. 

Junto a la capacidad de aprender, un elemento que se viene considerando como de crucial importancia para dar respuesta a 
los desafios actuales de las escuelas primarias es la capacidad de liderazgodc los profesores. En la revision que 
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recientemente realizara Smylie, (1998) encontro que los profesores que aspiraban a liderar, lo hadan por mejorar las 
cscuclas, que poseian las mismas caractensticas de personalidad que los no Hderes en relacion a asumir riesgos, que los 
profesores en esta posicion tienden a dedicar la mayor parte de su tiempo a desarrollar programas curriculares e 
instruccionales, organizando y desarrollando programas de formacion o desarrollando actividades administrativas, 
destacando el papel de los lideres en la puesta en marcha de programas de cambio, pero queda la duda de la permanencia 
de estos cambios en el tiempo. 

El liderazgo, tal como lo plantean las caractensticas anteriormente apuntadas pareceria que es una cualidad innata y 
peculiar de solo algunas personas. Sin embargo, si deseamos que la profesion docente avance nos parece que todos los 
profesores debemos convertimos en lideres. En este sentido coincidimos con Foster (1997) al entender que el liderazgo no 
es un papel o posicion separada y asumida bajo circunstancias especificas. Memos de pensar que el liderazgo es inherente 
al papel del profesor como profesional, que los profesores tienen una responsabilidad que cumplir. El compromiso con el 
liderazgo debe infiltrarse en los profesores que se preparan para entrar en la profesion. 

Y es inherente al liderazgo la capacidad de innovacion. Nos parece que una de las exigencias que la sociedad esta haciendo 
a los profesores de ensehanza primaria y secundaria es la capacidad de seguir aprendiendo e innovando Y para ello esta la 
actitud y el compromiso de los profesores con la mejoraque supone la innovacion. Y una de las principales innovaciones 
que se estan produciendo en la sociedad en general, y en lo que a nosotros respecta, la escuela, en particular son las 
Nuevas Tecnologias de la Informacion y la Comunicacion. Estas han introducido dentro de la educacion la posibilidad de 
disponer de recursos altamente orientados a la interaccion y el intercambio de ideas y materiales entre profesor y alumnos 
y de alumnos entre si. Las oportunidades que ofrecen para la cooperacion se extienden no solo al aprendizaje de los 
alumnos, sino tambien a la misma ensenanza, y engloban practicamente a todas las formas de comunicacion habituales en 
la educacion presencial. Este enfoque de aprendizaje cooperativo basado en soportes telematicos como Internet comienza a 
conocerse como educacion on-line, termino bajo el que se designa todo un conjunto de metodos que se ajustan 
especialmente a los principios del aprendizaje adulto, en el que el intercambio de la experiencia personal con relacion a un 
determinado contenido puede desempenar un papel relevante en el desarrollo colectivo (Marcelo, 2002). 

La simple incorporacion de las nuevas tecnologias en las escuelas, no obstante, no garantiza la efectividad en los 
resultados alcanzados, en el sentido de que la seleccion de medios y recursos interactivos y su incorporacion en un diseno 
global de entomo de teleformacion, deben estar sustentados sobre la base de una teoria del aprendizaje que los justifique y 
delimite. Uno de los aspectos mas llamativos de las nuevas tecnologias es su impacto en el Ambiente de Aprendizaje. Las 
nuevas tecnologias deberian incorporar un cambio en la forma de organizar la ensenanza y el aprendizaje. Y ello requiere 
de un profesor mas centrado en el aprendizaje que en la ensehanza, y por ello es necesario cuidar de la organizacion y 
disposicion de los contenidos de aprendizaje, asi como de la organizacion del aprendizaje de los alumnos mediante tareas 
individuales y en grupo, con un cuidado y permanente seguimiento por parte del tutor. Es un modelo de ensehanza 
centrado en problemas, en donde los alumnos no son meros receptores pasivos de datos estaticos, sino que deben resolver 
problemas utilizando para ello los contenidos adquiridos. Estos cambios se concretan segun Collins (1998) en: 

• Desde una ensehanza general a una ensehanza individualizada 



• Desde una ensehanza basada en la exposicion y explicacion a una ensehanza basada en la indagacion y la 
construccion 



• Desde trabajar con los mejores estudiantes a trabajar con todos 

• Cambios hacia estudiantes mas comprometidos con as tareas que realizan 

• Desde una estructura competitiva a una estructura cooperativa 

• Desde programas homogeneos a programas individualizados 

• Desde la primacia del pensamiento verbal a la integracion del pensamiento verbal y visual. 



Se esta demandando, por tan to un profesor entendido como un "trabajador del conocimiento”, disehador de ambientes de 
aprendizaje, con capacidad para rentabilizar los diferentes espacios en donde se produce el conocimiento. Y una profesion 
docente caracterizada por lo que Shulman (1998) ha denominado una comunidad de prdctica a traves de la que "la 
experiencia individual pueda convertirse en colectiva” (521). Una profesion que necesita cambiar su cultura profesional, 
marcada por el aislamiento y las dificultades para aprender de otros y con otros; en la que esta mal visto pedir ayuda o 
reconocer dificultades. 

La profesion docente ha ido arrastrando a lo largo su historia un deficit de consideracion social, basado, segun algunos en 
las caractensticas especificas de las condiciones de trabajo que la asemejan mas a ocupaciones que a "verdaderas" 
profesiones como la Medicina o el Derecho. Se ha querido comparar sistematicamente a la docencia con estas otras 
profesiones para ver si cumple las condiciones de "un conjunto de individuos que aplican un conocimiento cientifico 
avanzado para proporcionar un servicio a los clientesyse agrupan juntos mediante la pertenencia a un cuerpo 
profesional que asume la responsabilidad de controlar los promedios profesionales, y que les confer e beneflciosy puede 



imponer sanciones a los miembros** (Tomlinson, 1996). Y evidentemente, como Hoyle y John (1995) mostraban, la 
profesion docente, por sus especiales caracten'sticas, no cumple con estos estrictos y clasistas criterios. 

Con el discurrir de los tiempos y la implantacion de las refoimas educativas la profesion docente ha ido cambiando. Para 
algunos hacia una desprofesionalizacion, debido a la perdida progresiva de autonomia y control intemo. Para otros hacia 
una reprofesionalizacion, justificada por la necesidad de ampliar las tareas habitualmente asignadas a los docentes 
(Marcelo, 1999). En el primero de los sentidos se manifestaba David Hargreaves (1997) en un trabajo en el que reflexiona 
sobre el efecto que los cambios a los que nos hemos referido a lo largo de este articulo estan teniendo y van a tener en la 
profesion docente. Para este autor, las reclamaciones y luchas actuates en el sentido de una mayor profesionalizacion 
(mayor autonomia y autocontrol intemo de la profesion) de los docentes llega historicamente tarde. El avance imparable de 
la sociedad de la informacion, auspiciada como hemos visto por el uso de las Nuevas Tecnologias, va a configurar -segun 
este autor- un e scenario caracterizado por una "progresiva desprofesionalizacion: una sociedad del aprendizaje donde 
todo el mundo ensena y aprendey nadie es un experto" (D. Hargreaves, 1997: 1 9). 

Junto al concepto de profesionalizacion hemos hablado del profesionalismo, entendido en este caso como la capacidad de 
los individuos y de las instituciones en las que trabajan de desarrollar una actividad de calidad, comprometida con los 
clientes, y en un ambiente de colaboracion. Los estudios sobre profesionalismo han tornado en consideracion la necesidad 
de reprofesionalizar la funcion docente, y han percibido que la ampliacion de funciones es positiva, y muestra un sintoma 
claro de que los docentes son capaces de realizar funciones que van mas alia de las tareas tradicionales centradas en los 
alumnos y restringidas al espacio fisico del aula. Este nuevo profesionalismo, o profesionalismo extendido, segun la 
consideracion de A. Hargreaves y Goodson (1996), principalmente se concreta en las actuales demandas a los profesores 
para que trabajen en equipo, colaboren, planifiquen conjuntamente, pero que tambien incluye la realizacion de funciones 
mentoras o relacionadas con la formacion inicial de los profesores, asi como aspectos mds centrados en la formacion, 
como la formacion basada en la escuela. 

Un punto de vista critico respecto al profesionalismo extendido tiene que ver con el auge que se viene haciendo, y que 
nosotros hemos recogido a lo largo de este articulo, respecto a la necesidad de insertar en la profesidn docente normas de 
colaboracion, publicidad, compromiso compartido. Si bien hemos puesto de manifiesto que el aprendizaje organizativo 
requiere del trabajo planificado en comun, voces autorizadas (A. Hargreaves, 1992) nos llaman la atencion sobre como la 
colaboracion no puede convertirse en una nueva ortodoxia del cambio y de la mejora educativa. La colaboracion esta en la 
base de la reforma del curriculum y de las escuelas. Innovaciones tales como la ensenanza en equipo, la planificacion 
colaborativa, el '^coaching** entre companeros, el mentorazgo, o la investigacion-accion colaborativa son iniciativas que 
pueden ir a favor de un ambiente escolar que favorezca el aprendizaje. 

Algunos autores estan llamando la atencion acerca de la ironia de que mientras se esta vendiendo a los profesores y a las 
escuelas la idea de que deberian ser mas autonomos y responsables de las necesidades propias, a la vez se les esta 
transmitiendo como deben ser sus resultados y como debe abordar las prioridades nacionales para mejorar la competencia 
intemacional. Se supone que los profesores estan teniendo mas autonomia escolar precisamente en el mismo momento en 
que los parametros con los que se espera que trabajen y mediante los cuales seran evaluados, estan siendo cada vez mis 
serios y limitados (Little y McLaughlin, 1993;Smyth,1995;Day, 2001). 




Y, dado que la consideracion de profesionalidad no depende solo de la autopercepcion de los profesionales, sino tambien 
de la sociedad a la que sirve, y a los clientes con los que interactua, ha ido apareciendo desde finales de los anos noventa 
un movimiento que llama la atencion a la necesidad de establecer estdndares profesionales, piiblicos, asumidos por la 
profesion, como garantia y compromiso ante la ciudadania. En este sentido, y como han mostrado Yinger y Hendricks 
(2000), los estandares educativos constituyen una de las herramientas mas poderosas para la profesionalizacion de la 
ensenanza, ya que juegan un doble papel en el desarrollo de la profesion. En primer lugar, los estandares se constituyen en 
una buena manera de demostrar al publico y a los politicos que la profesion tiene suficientes procesos de control de 
calidad, controlando el acceso y el ejercicio de una practica eficaz, y asi ganar en legitimidad social. En segundo lugar, los 
estandares funcionan como parametros y orientaciones para el desarrollo del trabajo profesional, ya que pueden definir una 
practica eficaz en terminos de los resultados deseados. Y de esta manera, los estandares se convierten en la base para 
organizar la formacion y inicial y continua de los profesores, algo a lo que volveremos mas adelante. 

En la misma linea se ha venido mostrando Darling-Hammond (2001), para la que resulta fundamental que la ensenanza 
desarrolle su propio modelo de profesionalidad. Un modelo de profesionalidad que no pasa necesariamente por seguir el 
llevado a cabo por otras profesionales que han alcanzado el reconocimiento y prestigio a traves de un distanciamiento 
respecto de los clientes, asi como de un conocimiento altamente especializado. A diferencia de este modelo, la ensenanza 
debe insertarse en la comunidad y conectar con la vida de los alumnos, construyendo comunidades de aprendizaje 
relevantes y adecuadas para responder a las necesidades e intereses de los alumnos como ciudadanos con derecho a 
aprender. 

Y para caminar hacia ese reconocimiento, entiende que "la palanca politica mas importante de que se dispone para 
mejorar el proceso de ensenanza y aprendizaje es, probablemente, el desarrollo de unos estdndares profesionales que 
contemplen los aspectos mds importantes de la profesion docente" (Darling-Hammond, 2001: 391). Estos estandares, 
piiblicos y asumidos y desarrollados por la propia profesion pueden ser el eje vertebrador de los programas de formacion 
inicial y continua. Unos programa que, como veremos mas adelante, han estado organizados sin tener en cuenta el trabajo 
global del profesor en la escuela y olvidando que la formacion va dirigida a que los profesores aseguren el aprendizaje de 
los alumnos, ultimo criterio de calidad de la formacion. 




5. Nuevas formas de aprender 



Ya hemos comentado las dificultades con las que los profesores se encuentran para adaptarse a los cambios que se vienen 
produciendo en nuestras sociedades. Pero quizas el mayor cambio que tengan que dar aquellos que hoy en dia estdn 
ensenando, es reconsidera su compromiso con el aprendizaje y la formacion. Si es verdad que los profesores han seguido 
formandose de manera continuada, lo cierto hoy en dia es que esos esfuerzos no pueden seguir siendo iniciativas 
individuales y exclusivamente formales, sino que debemos tender a hacer realidad ese principio, meta u orientacion que es 
el del aprendizaje a lo largo de la vida. 

Aspin, Chapman, Hatton y Sawano (2001), editores del International Handbook of Lifelong Learning comenXan en la 
introduccion de la obra, que al aprendizaje a lo largo de la vida le ha llegado su momento. La idea de que la educacion y el 
aprendizaje son actividades y procesos que no empiezan y terminan con el inicio y final de la asistencia del individuo a las 
instituciones formales de educacidn es algo asumido desde la antiguedad cldsica. Sin embargo, hay algunos fenomenos que 
han hecho del aprendizaje no una eleccion sino una obligacidn. Como ellos comentan, 'Hstamos viviendo en una nueva era 
era en la que las demandas son tan complejas, tan multivariadas y tan cambiantes que la unica manera que seremos 
capaces de sobrevivir es mediante el compromiso con un proceso de aprendizaje individual, colectivoy global a lo largo 
de nuestra vida y para todos nosotros. " (xix), Un proceso de aprendizaje y formacion abierto que combina la formacion 
formal con la no formal e informal, la formacion individual con la grupal, 

El aprendizaje a lo largo de la vida no tiene que ver solo con la adquisicion de conocimientos sino con la actitud de 
desafio, capacidad de emprender, autoestima, innovacion no sdlo de los individuos de la cultura social (Leicester y Parker, 
2001). La European Lifelong Learning Initiative (Longworth, 2001) ha ofrecido una definicion de Lifelong learning: 

• El desarrollo del potencial humano a traves de 

• Un proceso continuamente apoyado 

• Que estimula y capacita a las personas 

• Para adquirir 

• Todos los conocimientos, valores, destrezas y conocimientos que requieren a lo largo de su vida 

• Y para aplicarlos 

• Con confianza, creatividad y disfrute 

• En todos los roles, circunstancias y ambientes. 

Una caracteristica del Aprendizaje a lo Largo de la Vida, y que desde mi punto de vista lo hace sumamente interesante, es 
que se entiende que TODOS podemos aprender, y que el aprendizaje no tiene por que estar limitado a las instituciones 
formales y tradicionales de formacion. En otras palabras, el aprendizaje que se considera de valor no solo es el aprendizaje 
formal, sino que el aprendizaje no formal e informal cobran la importancia que siempre han tenido aunque no hayan sido 
reconocidos. Aspin y Chapman (2001) comentaban al respecto que el conocimiento objetivo no solo se almacena en 
bibliotecas y esta presidido por autoridades que pertenecen a instituciones formales alojadas en escuelas y universidades. 
Tambien se pueden encontrar en aquellos lugares donde la gente es creativa y desarrolla respuestas imaginativas a 
preguntas y soluciones a problemas que se pueden proponer como hipotesis para ser falseadas en un discurso publico. 

El desarrollo y generalizacion de redes de profesores, la posibilidad de aprender con otros a la distancia, la creacion de 
escenarios abiertos y distribuidos que actualmente estdn siendo posibles gracias a las nuevas tecnologias de la informacion 
y la comunicacion, estan facilitando la visibilidad de esa forma de aprendizaje que hemos llamando informal. Y este hecho 
esta removiendo los tranquilos cimientos de las instituciones formales de acreditacion. Recogemos las palabras de Usher 
(2001) porque vienen a representar un indicativo del cambio que esta produciendo el aprendizaje a lo largo de toda la vida 
en la educacion y formacidn institucionalizadas. Dice este autorque *Las instituciones educativas estan encontrando 
dificultades crecientes en mantener el monopolio en la generacion y diseminacion del conocimiento. Cuando el 
conocimiento toma la forma de informacion, circula a traves de las redes que evaden el control de las instituciones 
educativas. Mas aun, las instituciones educativas, forman parte del mercado, dentro del negocio de la venta de 
conocimiento como mercancia y por lo tanto se reconstruyen a si mismas como empresas dedicadas al marketing de estas 
mercancias y a competir en el "negocio'' del conocimiento. No solo se convierten en parte del engranaje para producir 
personal para el post-ford ismo, sino que se espera que se comporten de una manera post-fordista" (2001 : 1 75). 
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En este escenario, el aprendizaje deja de ser un proceso pasivo para ser autodirigido, autointemalizado y autocontrolado. 
Asi, el aprendizaje no estd dirigido por el profesor sino centrado en los alumnos. Es un tipo de aprendizaje que contrasta 
con el aprendizaje formal. Hager (2001) establece lasdiferencias en los siguientes terminos: 



• El formador controla el aprendizaje formal mientras que es el alumno el que controla el aprendizaje informal: el 
aprendizaje formal se planifica mientras que el informal no. 

• El aprendizaje formal se desarrolla en instituciones educativas, en el trabajo y es ampliamente predecible. El 
aprendizaje informal no es predecible y no posee un curriculo formal. 



• Tanto en las instituciones educativas como en la formacion, el aprendizaje es explicito: se espera que el que ha 
asistido a formacion sea capaz de demostrarlo mediante examenes escritos, respuestas orales, etc. El aprendizaje 
informal generalmente es implicito, y en general el aprendiz no es consciente de lo que sabe, aunque sea 
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consciente de los resultados de ese aprendizaje. 



• En el aprendizaje formal se pone enfasis en la ensenanza, en el contenido y la estructura de lo que va a ser 
ensenado, mientras que en el aprendizaje informal el enfasis es en el que aprende. 

• En el aprendizaje informal el enfasis recae en los alumnos como individuos o en el aprendizaje individual, 
mientras que el aprendizaje informal a menudo es colaborativo. 

o El aprendizaje formal es descontextualizado, mientras que el aprendizaje informal es de naturaleza contextual izada 

• El aprendizaje formal toma forma en terminos de teon'a (o conocimiento) y despues practica (aplicacion de la 
teoria), mientras que el aprendizaje informal tiene que ver mas con conocer como se hacen las cosas. 

Estas ideas sobre el aprendizaje informal nos van a alumbrar mas adelante cuando revisemos la investigacion sobre el 
conocimiento de los profesores y hagamos referencia a las aportaciones de Donald Schdn sobre la epistemologia de la 
practica. For ahora quisieramos comentar algo que nos ha llamando la atencion a partir de la lectura del sugerente libro que 
sobre la Etica del Hacker ha publicado el filosofo finlandes Pekka Himanen (2001). Para este autor, el paradigma del 
aprendizaje en la sociedad del conocimiento tiene mucho que ver con la forma como los hackers aprenden, Recordemos 
que nos referimos al hacker como una persona con conocimientos informaticos y que desarrolla por si mismo y en 
colaboracion con otros, altemativas y desarrollos informaticos que desafian claramente a las grandes firmas comerciales de 
este sector. 

Pues bien, plantea Himanen, tomando como ejemplo a Linus Torval, autor del sistema operativo Linux, que el aprendizaje, 
en la sociedad del conocimiento, tiene que estar asociado con la pasion, con el interes por lo desconocido, por las 
preguntas mas que por las respuestas, por el apoyo de otros que conocen, por la resolucidn de problemas de manera 
colaborativa. Ese modelo de aprendizaje en el que lo que agrupa a las personas que aprenden no es la edad sino el 
problema a resolver, algo parecido a la Academia de Platon. Pero, como comenta Himanen, **La ironia es que la actual 
academia tiende a reproducir la estructura de aprendizaje emisor-receptor propia de los monasteries. La ironia 
generalmente se amplia cuando la academia construye una *universidad virtual*: el resultado es una escuela monacal 
compute rizada" (2001:76). 

6. que sabemos sobre como aprenden los profesores? 

Aprender a ensenar ha sido una constante en la preocupacion de los investigadores educativos en las ultimas decadas. 
Cientos de investigaciones y decenas de revisiones se han llevado a cabo para intentar comprender este proceso. Tanto en 
el tercer (Wittrock, 1986) como en el recientemente publicado cuarto Handbook of Research on Tbac/i mg (Richardson, 
2001) encontramos capitulos en los que se revisa y sintetiza el conocimiento sobre los profesores, su formacion y 
desarrollo. Igualmente en los Handbook of Research on Teacher Education (Houston, 1990; Sikula, 1996), pasando por el 
recientemente aparecido International Handbook of Teachers and Teaching Good y Goodson, 1997), el 

International Handbook of Educational Change (Hargreaves (1998), o el Handbook of Educational Psychology (Berliner y 
Calfee, 1996) se aborda de manera mas o menos amplia la investigacion sobre el aprendizaje de los profesores. Estos 
libros, asi como revisiones mas recientes, aparecidas en revistas especializadas, como la de Wilson y Berne (1999), 
Cochran-Smith y Little, 1999), Feiman (2001), Putnam y Borko (2000), Wideen, Mayer-Smith y Moon (1998) o Zeichner 
(1999) nos permiten configurar un panorama bastante actualizado respecto al conocimiento acumulado sobre el proceso de 
aprender a ensenar, asi como de sus luces y sombras. No puede ser intencion de este que les habla -o escribe- resumir lo 
que en centenares de paginas, otros investigadores ya han revisado. Pero tampoco dejaremos de lado la oportunidad de 
volver a recomponer este extenso puzzle que es el proceso de convertirse en un buen profesor. 

Y para este proposito creo que puede ser de utilidad dotamos de algun esquema conceptual que nos permita comprender la 
amplitud, complejidad y posibilidades de la investigacion sobre aprender a ensenar. El cubo que a continuacion 
presentamos pretende resumir en tres dimensiones las investigaciones que se han llevado a cabo. Una primera dimension 
tiene que ver con una diferenciacion en relacion con las fases por las cuales transcurre el proceso de aprender a ensenar. 
Asi, podemos diferenciar entre la formaci6n inicial, entendida como el transito por parte del profesor en formacion por un 
itinerario formativo disenado especificamente para dotarle de los conocimientos, habilidades y disposiciones necesarias 
para ejercer su tarea docente. Una segunda fase la constituyen los primeros anos de ensenanza, denominados como anos 
de iniciacion, induccion o inserci6n profesional. Una ultima fase de analisis del proceso de aprender a ensenar se refiere a 
aquellos profesores que han generado su propio repertorio profesional y que avanzan a traves de experiencias de 
desarrollo profesional. 
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Figura 1. Tres dimensiones para explicar el proceso de aprender a enseflar 



Una segunda dimension que consideramos importante incorporar en cualquier revision sobre la investigacion en el proceso 
de aprender a ensenar tiene que ver con los temas. Memos hecho un esfuerzo de sintesis diferenciadora para reducir a ocho 
los temas posibles. Nos basamos en el trabajo que hace ya algunos anos realizaron Katz y Rath (1985). Los temas en 
cuestion se refleren a los profesores, sus conocimientos, creencias, disposiciones, actitudes, autoeflcacia percibida, etc., los 
contenidos de la formacion, los metodos y estrategias formativas, los formadores de profesores, las practicas, asi como el 
ambiente y la evaluacion. Estos temas, evidentemente se mezclan en muchas investigaciones pero pueden servir como 
marco de referencia para clasificar la investigacidn. 



Y una tercera dimension tiene que ver con el enfoque adoptado por los investigadores para abordar los temas 
seleccionados. Aun a riesgo de simplificar de manera excesiva, nos parece adecuada una diferenciacion entre enfoque 
cuantitativos y cualitativos. Esta ha sido una opcion tambien tomada por Richardson y Placier (2001), en su revision sobre 
la investigacion en relacion con el cambio de los profesores. Estos autores, basandose en un trabajo de Chin y Benne, 
diferencian dos enfoques principales respecto al cambio, el aprendizaje, desarrollo, socializacion de los profesores: un 
enfoque empirico-analitico que contempla a los profesores como receptores y consumidores de propuestas de cambio que 
vienen de los asesores, politicos, educadores, investigadores. Se trata de un proceso lineal de cambio originado fuera de las 
aulas, que aporta una nueva idea, forma de pensar o programa de ensenanza basado normalmente en la investigacion o en 
la teorias. A los profesores se les dice de que va el cambio, se le demuestra y se espera que lo aplique. En esta vision el 
cambio es complejo. Una segunda orientacion denominada normativa-reeducativa, parte de un movimiento mas amplio de 
estudio fenomenologico sobre la forma como las personas dan sentido y contribuyen a la situacion en la que trabajan. Los 
enfoques de cambio, desde esta perspectiva asumen la necesidad de reflexion profunda sobre las creencias y las practicas y 
en el que el dialogo se convierte en una practica habitual. 



En la misma linea, Sykes (1998) llega a identificar tres imagenes que ilustran las caracteristicas actuales de como se 
investiga el proceso de aprender a ensenar: a) el profesor entendido como consumidor dentro de un mercado 
cuasirregulado, estructurado por unos servicios que se proporcionan burocraticamente. En este caso la formacion se 
planifica, se regula por estructuras jerarquicamente organizadas; b) el profesor entendido como un artesano independiente 
construyendo conocimiento, habilidades y materiales en un enfoque de oficio. Este modelo corresponde al profesor 
trabajando solo en clase que acumula sabiduria y saber hacer, analizandose como se adquiere y renueva el conocimiento; 
c) Una tercera vision asume al profesor como profesional que orienta su trabajo de acuerdo a las normas comunes. 

Esta diferencia en los enfoques de investigacion, que ya Fenstermacher (1994) utilize para agrupar su revision y cn'tica 
hacia la investigacidn sobre el conocimiento de los profesores, ha sido empleada recientemente por Cochran-Smith y Lytle 
(1999). Fenstermacher se planteaba las preguntas de ^que se sabe acerca de la ensenanza eficaz? ^que conocen los 
profesores? ^que conocimiento es esencial para la ensenanza? y ^quien produce conocimiento acerca de la ensenanza? 



como forma de diferenciar y clasificar enfoques distintos de investigacion sobre aprender a ensenar. Evidentemente, detras 
de estas preguntas estan los enfoques cuantitativo (presagio*producto y proceso producto) y los enfoques cualitativos 
(mediacional cognitivo, ecologico). Cochran-Smith y Lytle (1999), siguiendo la idea de Fenstermacher ban venido a 
clasificar la investigacion sobre aprender a ensenar tomando como criterio las relaciones que establece entre el 
conocimiento que se produce y su aplicacion en la practica de la ensenanza. Asi, diferencian entre: 

• Conocimiento para la practica: Esta primera concepcion entiende en que la relacion entre conocimiento y 
practica es aquella en que el conocimiento sirve para organizar la practica, y por ello, conocer mas (contenidos, 
teorias educativas, estrategias instruccionales) conduce de forma mas o menos directa a una practica mas eficaz. 

El conocimiento para ensenar es un conocimiento formal, que se deriva de la investigacion universitaria, y es al 
que se refieren los teoricos cuando se habla de que la ensenanza ha generado un cuerpo de conocimiento diferente 
al conocimiento comun. La practica, desde esta perspectiva, tiene mucho que ver con la aplicacion del 
conocimiento formal a las situaciones practicas. 

• Coiiociniiento en la practica: El enfasis de la investigacion sobre aprender a ensenar ha sido la busqueda del 
conocimiento en la accion. Se ha estimado que lo que los profesores conocen esta implicito en la practica, en la 
reflexion sobre la practica, en la indagacion practica y en la narrativa de esa practica. Un supuesto de esta 
tendencia es que la ensenanza es una actividad incierta y espontanea, contextualizada y construido en respuesta a 
las particularidades de la vida diaria en las escuelas y las clases. El conocimiento esta situado en la accidn, en las 
decisiones y juicios que toman los profesores. Este conocimiento se adquiere mediante la experiencia y la 
deliberacion y los profesores aprenden cuando tienen oportunidad de reflexionar sobre lo que hacen. 

• Conocimiento de la practica. Esta ultima tendencia se incluye dentro de la li'nea de investigacion cualitativa, pero 
cercana al movimiento denominado del profesor como investigador. La idea de la que parte es que en la 
ensenanza no tiene sentido hablar de un conocimiento formal y otro conocimiento practico, sino que el 
conocimiento se construye colectivamente dentro de comunidades locales, formadas por profesores trabajando en 
proyectos de desarrollo de la escuela, de formacion o de indagacion colaborativa (Cochran -Smith y Lytle, 1999). 

La investigacion sobre aprender a ensenar ha sido abordada con diferentes objetivos y final idades. En la revision que sobre 
los "modos de indagacion" en formacion del profesorado realizaron Lee y Yarger (1996) clasifican de una manera mas 
amplia las tendencias que nosotros hemos resumido anteriormente. Estos autores describen los diferentes modos de 
indagacion (entendidos una estrategia intelectual seleccionada por el investigador para plantear cuestiones de investigacion 
en formacion del profesorado) diferenciando entre los siguientes enfoques: 

• Investigacion experimental y cuasi-experimentah han sido, segun estos autores, el modo tradicional de 
investigacion en formacion del profesorado. Dentro de la tradicion conductista las investigaciones experimentales 
se han utilizado para formar a profesores para el dominio de estrategias de ensenanza especiTicas, bien sean 
conductuales, cognitivas, inductivas. Un buen ejemplo de este tipo de investigacion ha sido el analisis de 
interaccion y la microensenanza. 

• Los estudios correlacionales son, a partir de la revision de Lee y Yarger muy escasos en la investigacion sobre 
formacion del profesorado. En algunos casos se han contrastado a traves de analisis de regresion las caracteristicas 
del profesor que contribuyen al rendimiento de los alumnos (paradigma presagio-producto) 

• Investigacion descriptiva tipo "survey": Este tipo de investigacion se ha utilizado con amplitud por las 
instituciones de formacion del profesorado intentando conocer las caracteristicas de los candidatos, la valoracion 
de los componentes de los programas, etc. 

• Investigacidn con estudio de caso\ Recientemente este modelo de investigacidn se ha convertido en uno de los 
modos de investigacion mas utilizados. Ello es asi por las caracteristicas de la formacion del profesorado y la 
necesidad de comprender los contextos. Se ha investigado el impacto de los programas de formacion del 
profesorado en los conocimientos, habilidades y disposiciones de los profesores en formacion. En la linea de la 
investigacion con estudio de caso se situa la indagacion narrativa y la investigacion del profesor 

• Investigacion etnogrdfica: Considerando que la formacion del profesorado es una actividad compleja, la etnografia 
puede ser una herramienta de utilidad al describir y comprender temas como los grupos, la cultural la 
participacion. Sin embargo hay pocos estudios etnograficos en formacion del profesorado y los que se describen 
asi, lo que utilizan son tecnicas etnograficas. 

• Investigacion historica: Este modo de investigacion ha sido permanente a lo largo de la investigacion sobre 
formacion del profesorado indagando temas como la historia de la ensenanza y de los profesores, contratos, 
condiciones de trabajo, curriculo de la formacion del profesorado, etc. 

Como hemos visto anteriormente, la investigacion que se ha desarrollado en el vasto campo que es la formacion del 
profesorado ha ido avanzando mediante la incorporacion de nuevos enfoques que necesariamente han venido a 
complementar visiones mas reduccionistas. Hoy en dia conviven las investigaciones que utilizan enfoques cualitativos y 
cuantitativos, aunque, como senalaban Lee y Yarger (19996), sea el estudio de caso la tabla de salvacion de muchos 
investigadores. 

Pero sea un enfoque cualitativo o cuantitativo, algo que ha caracterizado a la investigacion sobre formacion de profesorado 
ha sido que los temas de investigacion abordados (aquellos que hemos seleccionado para nuestro modelo) rara vez se han 
abordado de manera transversal. Quiero decir, que las investigaciones se han centrado en uno o dos temas pero que en 
escasas ocasiones se han integrado analisis de programas de formacion en los que intervengan los profesores, los 
formadores, lo contenidos, los medios, etc. Por ello vamos a revisar algunos de los temas que mas atencion han recibido, 
sabiendo que esta revision necesariamente debe ser selectiva y limitada al espacio que ya nos va quedando en este articulo. 





6.1. Los profesores como objeto de investigaci6n 

Uno de los temas sobre los que se ha vertido m^s tinta y dedicado mas horas de estudio tiene que ver -no podia ser de otra 
forma- con la preocupacion por conocer las caracten'sticas de los sujetos que deciden dedicarse profesionalmente a la 
docencia. Y esta preocupacion abarca desde el inicio de la formacion inicial hasta la jubilacion. Y no se piense que la 
jubilacion deja de tener interes para los investigadores. Recuerdese uno de los escenarios que hemos descrito 
anteriormente, elaborados por la OCDE (2001 ) que nos llamaba la atencion acerca de la posibilidad de necesitar recurrir a 
profesores jubil ados para que cubran los espacios dejados por otros profesores que abandonaban la ensenanza. 

Las creencias de los profesores 

La investigacion sobre los profesores como objeto de estudio puede clasificarse en los tres momentos que hemos descrito 
en la Figura 1 : formacidn inicial, insercion profesional y desarrollo profesional. En la formacion inicial ha habido una 
especial preocupacion por el analisis de las creencias que los profesores en formacion traen cuando inician su andadura 
profesional. Se ha entendido que las creencias son como proposiciones, premisas que mantienen las personas acerca de lo 
que consideran verdadero. Las creencias, a diferencia del conocimiento proposicional no requieren una condicion de 
verdad contrastada, y cumplen dos funciones en el proceso de aprender a ensefiar. En primer lugar, las creencias influyen 
en la forma como aprenden los profesores; y en segundo lugar, las creencias influyen en los procesos de cambio que los 
profesores puedan intentar (Richardson, 1 996). 

La literatura de investigacion sobre aprender a ensefiar ha identificado tres categorias de experiencias que influyen en las 
creencias y conocimientos que los profesores desarrollan sobre la ensenanza: 

• Experiencias personales: Incluyen aspectos de la vida que determinan una vision del mundo, creencias hacia uno 
mismo y en relacion con los demas, ideas acerca de las relaciones entre la escuela y la sociedad, asi como sobre la 
familia y la cultura. La procedencia socioeconomica, etnica, el sexo, religion pueden afectar las creencias acerca 
del aprender a ensefiar. 

• Experiencia con el conocimiento formal: El conocimiento formal, entendido como aquello sobre lo que debe 
trabajarse en la escuela. Las creencias acerca de la materia que se ensefia asi como la forma de ensefiarla 

• Experiencia escolar y de aula: Incluye todas aquellas experiencias como estudiante que contribuyen a formar una 
idea acerca de que es ensefiar y cual es el trabajo del profesor. 

De entre los hallazgos mas divulgados esta el hecho de que las creencias que los profesores en formacion traen consigo 
cuando inician su formacion inicial afectan de un manera directa a la interpretacion y valoracion que los profesores hacen 
de las experiencias de formacion del profesorado. Esta modalidad de aprender a ensefiar se produce a traves de lo que se 
ha denominado aprendizaje por la observacion (Lortie, 1975). Aprendizaje que en muchas ocasiones no se genera de 
manera intencionada, si no que se va adentrando en las estructuras de cognitivas -y emocionales- de los futuros profesores 
de manera inconsciente, llegando a crear expectativas y creencias dificiles de remover. 

Hay que hacer referencia a Pajares (1992) como uno de los investigadores que ha realizado mas contribuciones en relacion 
con el analisis de las creencias. Llamo la atencion respecto a la dispersion semantica que ha caracterizado a esta Hnea de 
investigaciones, en las que se han utilizado terminos como: creencia, actitud, valores, juicios, axiomas, opiniones, 
ideologia, percepciones, concepciones, si sterna conceptual, preconcepciones, disposiciones, teorias implicitas, teorias 
explicitas, teorias personales, procesos mental es intemos, reglas de la practica, principios practicos, etc. Esta dispersion 
semantica ha producido que los resultados de investigaciones no puedan compararse por no compartir un mismo marco 
conceptual. Pajares ha diferenciado entre conocimiento y creencias, poniendo de manifiesto que las creencias, a diferencia 
del conocimiento poseen una clara connotacidn afectiva y evaluativa: conocimiento de un tema se diferencia de los 

sentimientos que tengamos sobre ese tema, de la misma manera que se diferencia entre autoconcepto y autoestima, entre 
conocimiento de si mismo y sentimiento del valor propio" (Pajares, 1 992:309). 

Desde esta diferenciacibn, las investigaciones han venido mostrando que los profesores en formacion entran en el 
programa de formacion con creencias personales acerca de la ensenanza, con imagenes de buen profesor, imagen de si 
mismos como profesores y la memoria de si mismos como alumnos. Estas creencias e imagenes personales generalmente 
permanecen sin cambios a lo largo del programa de formacion y acompana a los profesores durante sus prbcticas de 
ensenanza (Kagan, 1992; Feiman, 2001, Wideen, Mayer-Smith y Moon, 1998). 

Pajares (1992) (Nota 1) sintetizb los resultados de la investigacion de las creencias de los profesores en los siguientes 
principios: 

Las creencias se forman en edad temprana y tienden a perpetuarse, superando contradicciones causadas por la razon, el 
tiempo, la escuela o la experiencia. 

1 . Los individuos desarrollan un sistema de creencias que estructura todas las creencias adquiridas a lo largo del 
proceso de transmision cultural. 

2. Los sistemas de creencias tienen una funcion adaptativa al ayudar al individuo a definir y comprender el mundo y 
a si mismos. 

3. Conocimiento y creencias estbn interrelacionados, pero el caracter afectivo, evaluativo y episodico de las creencias 




se convierten en un filtro a traves del cual todo nuevo fenomeno se interpreta. 

4. Las subestructuras de creencias, como son las creencias educativas, se deben comprender en terminos de sus 
conexiones con las demas creencias del sistema. 

5. Debido a su naturaleza y origen, algunas creencias son mas indiscutibles que otras. 

6. Cuanto mas antigua sea una creencia, mas dificil es cambiarla. Las nuevas creencias son mas vulnerables al 
cambio. 

7. El cambio de creencias en los adultos es un fenomeno muy raro. Los individuos tienden a mantener creencias 
basadas en conocimiento incompleto o incorrecto. 

8. Las creencias son instrumentales al definir tareas y al seleccionar los instrumentos cognitivos con los cuales 
interpretar, planificar, y tomar decisiones en relacion a estas tareas; por lo tanto juegan un papel crucial al definir 
la conducta y organizar el conocimiento y la informacion. 



Pero al igual que desarrollamos conocimientos y creencias generales acerca de la ensenanza, de los alumnos, la escuela o 
el profesor, la materia que ensenamos o pretendemos ensenar no se queda al margen de nuestras concepciones. La forma 
como conocemos una determinada disciplina o area curricular afecta a como la ensenamos. Existen multiples evidencias 
que nos muestran ciertos "arquetipos" que los profesores en formacion tienen sobre la disciplina que estudian, ya sea esta 
matematicas, lengua o educacion fisica. Preguntas como i,que son y para que sirven las matematicas, lengua, educacion 
fisica, etc.? son necesarias de plantear cuando pretendemos "partirde lo que el alumno ya sabe*\ Tomando el contenido 
que se ensena y se aprende como argumento de la indagacion, podemos encontrar diferencias en el comportamiento 
observable de profesores en funcion del dominio que posean del contenido que ensenan (Onofre, 2001). 



El conocimiento de los profesores 

Desde la aportaciones de Shulman (1986) o Schon (1983) en las que llamaron la atencion acerca de que la ensenanza no 
era una actividad tecnica, sino que se regia por un tipo de conocimiento ligado a la accion, a la practica, ban sido miles las 
investigaciones que se ban desarrollado en el campo de la formacion del profesorado en las que se ba intentado conocer 
que conocen los profesores, como began a conocerlo y -lo que resulta mas importante- como podemos mejorar el 
conocimiento de los profesores. Ya bemos becbo referenda a las preguntas formuladas por Fenstermacber (1994) que nos 
sirven para entender las respuestas investigadoras que se ban venido dando a lo largo de estos anos. Y a partir de aqui 
identificamos dos tipos de trabajo: aquellos que pretenden describir que conocen los profesores (tipos de conocimiento, 
forma de adquisicion) y aquellos otros que buscan determinar que deben conocer los profesores para ensenar de manera 
eficaz. 

En esta ultima linea de discurso se situan Ball y Coben (1999) cuando plantean los conocimientos que los profesores 
deberian poseer: 

1 . Deberian comprender bien la materia que ensenan, de forma muy diferente a la que aprendieron como estudiantes. 
Necesitan comprender en profundidad el contenido, asi como la forma en que este se conecta con la vida cotidiana 
para resolver problemas. 

2. Ademas de conocer la materia que ensenan, los profesores deben conocer acerca de los alumnos, como son, que 
les interesara, etc. 

3. Los profesores necesitan aprender que conocer a los alumnos no es simplemente conocer a alumnos 
individualmente. Debido a que los profesores ensenan a alumnos de procedencia muy diversa deben conocer 
acerca de las diferencias culturales, incluyendo el lenguaje, clase social, familia y comunidad. 

4. Los profesores necesitan tambien conocer sobre didactica, modelos de ensenanza , asi como sobre la cultura del 
aula. 

Una de las aportaciones que siguen siendo utilizadas para comprender el conocimiento de los profesores es la desarrollada 
por Grossman (1990) que caracterizo el conocimiento de la siguiente forma: 
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En primer lugar, Grossman destaca la necesidad de que los profesores posean un conocimiento pedagogico general, 
relacionado con la ensenanza, con sus principios generales, con el aprendizaje y los alumnos, asi como con el tiempo de 
aprendizaje academico, el tiempo de espera, la ensenanza en pequenos grupos, la gestion de clase, etc. Tambien incluye el 
conocimiento sobre tecnicas didacticas, estructura de las clases, planificacion de la ensenanza, teorias del desarrollo 
humano, procesos de planificacion curricular, evaluacion, cultura social e influencias del contexto en la ensefianza, historia 
y filosofia de la educacibn, aspectos legales de la educacion, etc 

Junto al conocimiento pedagogico, los profesores ban de poseer conocimiento de la materia que ensenan. Saber sobre la 
asignatura que ensenamos, tener un manejo fluido de la disciplina que impartimos es una zona ineludible del oficio 
docente. Al respecto, Buchmann nos senala que "conocer algo nos permite ensenarlo; y conocer un contenido con 
profundidad significa estar mentalmente organizado y bien preparado para ensenarlo de una forma general" (1984:37). 
Cuando el formador no posee conocimientos adecuados de la estructura de la disciplina que esta ensenando, puede 
representar erroneamente el contenido a los alumnos. El conocimiento que los formadores poseen del contenido a ensenar 
tambien influye en el que y el como ensenan. 

El Conocimiento del Contenido incluye diferentes componentes, de los cuales dos son los mas representativos: 
conocimiento sintactico y sustantivo. El Conocimiento Sustantivo se constituye con la informacion, las ideas y los topicos 
a conocer, es decir, el cuerpo de conocimientos generales de una materia, los conceptos especiTicos, definiciones, 
convenciones, y procedimientos. Este conocimiento es importante en la medida en que determina lo que los profesores van 
a ensenar y desde que perspectiva lo haran. Por ejemplo, en Historia, el marco de an^lisis cultural, politico o ideologico 
que se escoja, puede determinar el que se ensena, y como se ensena. El Conocimiento Sintactico del contenido completa al 
anterior, y se encama en el dominio que tiene el formador de los paradigmas de investigacion en cada disciplina, del 
conocimiento en relacion con cuestiones como la validez, tendencias, perspectivas e investigacion en el campo de su 
especialidad. En Matematica, seria la distincion entre convencion y construccion logica; en Historia incluiria las diferentes 
perspectivas de interpretacion de un mismo fenomeno; en Ciencias Naturales, el conocimiento sobre el empirismo y el 
metodo de investigacibn cientifica, etc. 

El Conocimiento Didactico del Contenido aparece como un elemento central de los saberes del formador. Representa la 
combinacion adecuada entre el conocimiento de la materia a ensenar y el conocimiento pedagogico y didactico referido a 
como ensenarla. En los ultimos anos, se ha venido trabajando en diferentes contextos educativos para clarificar cuales son 
los componentes de este tipo de conocimiento profesional de la ensenanza. El Conocimiento Didactico del Contenido, 
como linea de investigacion, representa la confluencia de esfuerzos de investigadores didacticos con investigadores de 
materias especiTicas preocupados por la formacion del profesorado. El Conocimiento Didactico del Contenido nos dirige a 
un debate en relacion con la forma de organizacion y de representacion del conocimiento, a traves de analogias y 
metaforas. Plantea la necesidad de que los profesores en formacion adquieran un conocimiento experto del contenido a 
ensenar, para que puedan desarrollar una ensenanza que propicie la comprension de los alumnos. 



Shulman (1992) manifestaba la necesidad de que los profesores construyeran puentes entre el significado del contenido 
curricular y la construccion realizada por los alumnos de ese significado. Este prestigioso investigador, afirma que "los 
profesores llevan a cabo esta hazana de honestidad intelectual mediante una comprension profunda, flexible y abierta del 
contenido; comprendiendo las dificultades mbs probables que tendran los alumnos con estas ideas [...] ; comprendiendo las 
variaciones de los metodos y modelos de ensenanza para ayudar a los alumnos en su construccion del conocimiento; y 
estando abierto a revisar sus objetivos, planes y procedimientos en la medida en que se desarrolla la interaccion con los 
estudiantes. Este tipo de comprensibn no es exclusivamente tecnica, ni solamente reflexiva. No es sblo el conocimiento del 



contenido, ni el dominio generico de metodos de ensenanza. Es una mezcla de todo lo anterior, y es principalmente 
pedagogico" (Shulman, 1992:12). 

Un cuarto componente del conocimiento que han de adquirir los formadores y los profesores, radica en el donde y a quien 
se ensena. Los profesores han de adaptar su conocimiento general de la materia a los alumnos y a las condiciones 
particulares de la escuela. Yinger ha planteado la dimension ecologica del conocimiento, entendiendo que el conocimiento 
no existe en los individuos, sino en las relaciones que se producen entre estos y el ambiente en que se desarrollan. La vida 
del aula, en este sentido *’esta constituida por los sistemas culturales, fisicos, sociales, historicos, y personales, que existen 
tanto dentro como fuera de la clase [...] La responsabilidad del profesor en la clase consiste en comprender las 
conversaciones que estan ocurriendo dentro y entre todos los sistemas y reconocer cuales son apropiados para la actividad 
de la clase. El profesor actiia como guia y sujeto que traslada la estructura, la accion, y la informacion incluida en cada 
sistema" (Yinger, 1991:31). 



Lo conientado hasta ahora nos plantea la idea de que el conocimiento de los profesores se construye en interaccion con las 
experiencias anteriores y actuales, en contacto con la practica, asi como en relacion con las actividades de formacion en las 
que los profesores participan. Pero ademas de su caracter construido^ el conocimiento de los profesores tiene otras 
dimensiones a las que recientemente Putnam y Borko se ha referido en algunos articulos de interes (Putnam y Borko, 

2000a y b). Elios hacen una propuesta, desde mi punto de vista muy interesante que amplia las estrechas miras de la 
investigacion individualista que ha sido aquella que ha tratado de analizar el conocimiento de profesores aisladamente. 
Proponen considerar que el conocimiento de los profesores, ademas de construido, debe ser entendido como un 
conocimiento contextualizado, social y distribuido. 

Se ha venido entendiendo que la formacion y el aprendizaje del profesor puede producirse, como hasta ahora hemos 
comentado, de forma relativamente autonoma y personal. Pero poco a poco ha ido ganando terreno la teorias que entienden 
la formacion como un proceso que ocurre no de forma aislada sino dentro de un espacio iniersubjetivo y social (Pastre, 
1994). Asi, aprender a ensenar no deberia entenderse s61o como un fenomeno aislado, sino basicamente como una 
experiencia que ocurre en interaccion con un contexto o ambiente con el que el individuo interacciona. Es la tesis del 
enfoque sociocultural del aprendizaje que establece que la actividad cognitiva del individuo no puede estudiarse sin tener 
en cuenta los contextos relacionales, sociales y culturales en que se lleva a cabo (Wertsch, 1993). 



Esta idea ha sido asumida por Yinger (1991), quien propone utilizar el concepto "working knowledge", para hacer ver que 
el conocimiento se produce en diferentes situaciones. Este autor habla que "Estamos comenzando a ver que los enfoques y 
las concepciones cambian desde lo individual a lo cooper at ivo y comunitario, de centrarse en la informacion a hacerlo en 
la accion, desde supuestos mecanicistas a supuestos organismicos, desde la medicion a la narrativa, desde la abstraccion 
a la concrecion, desde las operaciones a las conversaciones" (Yinger, 1 991 :5). 

Resulta de gran interes y proyeccion este enfoque, puesto que pone de manifiesto que la unidad de andlisis del proceso de 
aprender a ensenar son los procesos de interaccion social, llamando la atencion al analisis conversacional: "la 
conversacion se considera el contexto natural en el que las habilidades cognitivas de los sujetos se transforman en 
acciones y se construyen en torno a la interaccion interindividual" (Schubauer-Leoni y Grossen, 1993:452). Asi, los 
grupos sociales crean lo que se ha venido en llamar "comunidades discursivas" que comparten formas de pensar y de 
comunicarse. Comunidades que establecen redes y que sirven para compartir, intercambiar, situarse en el mundo, recibir 
apoyo, etc. (Lieberman y Grolnick, 1998). 

Completando la idea anterior, se ha venido avanzando en entender que el conocimiento en general y el pedagogico en 
particular no puede comprenderse al margen del contexto en el que surge y al que se aplica. McLellan afirma que "el 
modelo de conocimiento situado se basa en el principio de que el conocimiento estd situado contextualmente, y estd 
influido fundamentalmente por la actividad, el contexto y la cultura en la que se utiliza" (1996:6). No cabe, por tanto, 
diferenciar de manera radical el conocimiento que se adquiere y el contexto en el que ese conocimiento se utiliza: 
"tXconocimiento sobre la ensenanza no puede aprenderse de forma independiente de las situaciones en las que este se 
w////zfl"(Marx et al. 1998:34). Como consecuencia de entender el conocimiento de manera contextualizada se nos plantea 
con dureza la pregunta de ^que utilidad tiene para la formacion inicial del profesorado un conocimiento expresado de 
forma proposicional, sin vinculos con la situacion o contexto donde pueda contrastarse o aplicarse? 

Hablamos por tanto de la capacidad de transferencia de aprendizaje que nuestros profesores en formacion tienen de los 
conocimientos que la institucion de formacion considera basicos para aprender a ensenar. Tambien nos plantea la 
necesidad de revisar la forma como se presenta, comunica y construye ese conocimiento. 

Una ultima caracteristica del conocimiento que caracteriza el aprender a ensenar, es que no reside en una sola persona, sino 
sino que esta distribuido, entre individuos, grupos y ambientes simbolicos y fisicos (Putnam y Borko, 2000b). Se asume 
la idea de que para el desarrollo de tareas complejas, y aprender a ensenar evidentemente lo es, ninguna persona posee la 
totalidad de conocimientos y habilidades de forma individual. 
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Admitir este principio nos lleva a entender que es el trabajo en equipo lo que conduce a un mejor uso del conocimiento, lo 
que lleva a mejorar la capacidad de resolucion de problemas. Como Senge plantea en su sugerente libro titulado La Quinta 
Disciplina: "Ya no basta con tener una persona que aprenda para la organizacion... Ya no es posible "otear el panorama" 
y ordenar a los demds que sigan las ordenes del "gran estratega". Las organizac tones que cobrardn relevancia en el 
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futuro serdn las que descubran como aprovechar el entusiasmo y la capacidad de aprendizaje de la gente en todos los 
niveles de la organizacion” (Senge, 1 992: 11-12). 



La idea del conocimiento distribuido se ha visto impulsada por el impacto de las Nuevas Tecnologias, principalmente 
Internet. La posibilidad de que los profesores puedan acceder a conocimientos y contactos personales con profesores 
distantes geograficamente, la posibilidad de pertenencia a "comunidades virtuales" esta ampliando las posibilidades de lo 
que se entiende por aprender a ensenar (Marcelo, 2002). 

Los profesores pasan por diferentes fuses en el proceso de aprender a ensenar 

Esta Hnea de investigacidn es una de las pocas que ha presentado resultados transversales en alguno de los ejes que hemos 
descrito en la Figura 1. Se trata de la investigacion que pretende establecer diferencias entre profesores en funcion de la 
edad, asi como de lo que se ha denominado "expertise". Y esta evolucion, salvo en casos excepcionales, se ha comenzado a 
analizar a partir del primer ano de experiencia docente. Por una parte tenemos aquellos estudios que intentan comprender el 
proceso de convertirse en experto, y por otro aquellos otros estudios que analizan que hacen y que caracteriza a los 
profesores expertos. Dentro de estos estudios ha sido clasico el contraste entre los profesores expertos y principiantes. Hay 
que senalar que cuando hablamos del profesor experto nos referimos no solo a un profesor con, al menos, cinco anos de 
experiencia docente, sino sobre todo a una persona con un "elevado nivel de conocimiento y destreza, cosa que no se ad- 
quiere de forma natural, sino que requiere una dedicacion especial y constante" (Bereiter y Scardamalia, 1 986: 10). Asi, la 
competencia profesional del profesor experto no se consigue a traves del mero transcurrir de los anos. No es totalmente 
cierto, como senala Berliner, que la simple experiencia sea el mejor profesor. Si no se reflexiona sobre la conducta no se 
Ilegara a conseguir un pensamiento y conducta experta (Berliner, 1986). 

Segun Bereiter y Scardamalia, los sujetos expertos -en cualquiera de las areas- tienen en comun las siguientes 
caracteristicas: complejidad de las destrezas, es decir, el experto realiza sus acciones apoyandose en una estructura 
diferente y mas compleja que la del principiante, ejerciendo un control voluntario y estrategico sobre las partes del proceso, 
que se desarrolla mas automaticamente en el caso del principiante. En segundo lugar, figura la cantidad de conocimiento 
que el experto posee en relacion al principiante, que posee menos conocimientos. En tercer lugar senalan la estructura del 
conocimiento. Para Bereiter y Scardamalia, "los principiantes tienden a tener lo que podemos describir como una 
estructura de conocimiento 'superficial', unas pocas ideas generales y un conjunto de detalles conectados con la idea 
general, pero no entre si Los expertos, por otra parte, tienen una estructura de conocimiento profunda y multinivel, con 
muchas conexiones inter e intranivel” (\9S6: 12). La ultima caracteristica que diferencia a expertos de principiantes es la 
representacion de los problemas: el sujeto experto atiende a la estructura abstracta del problema y utiliza una variedad de 
tipos de problemas almacenados en su memoria. Los principiantes, por el contrario estan influidos por el contenido 
concreto del problema y, por tanto, tienen dificultades para representarlo de forma abstracta (Marcelo, 1999). 

Otro de estos estudios fue el que llevaron a cabo Sternberg y Horvarth (1995) que mostro que "La primera diferencia 
pertenece al dominio del conocimiento. Los expertos poseen mayor conocimiento y lo emplean de forma mas eficaz para 
resolver problemas que los principiantes. La segunda diferencia tiene que ver la eficacia en la resolucion de problemas. 
Los expertos hacen mas en menos tiempo (en su dominio de expertise) que los principiantes. La tercera diferencia tiene 
que ver con la intuicidn. Es mas probable que los expertos lleguen a las soluciones mas apropiadas. " (10). 



Pero dejando aparte los estudios sobre profesores expertos, cabe destacar las investigaciones que han intentado establecer 
etapas evolutivas dentro de la carrera profesional de los profesores. Huberman, Thompson y Weiland (1998) sintetizaron 
esta Hnea de investigacion, ofreciendo un cuadro comparativo respecto de seis estudios: 
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Como puede comprobarse por el compendio realizado por Huberman, Thompson y Weiland, existe sintonia entre los 
investigadores a la hora de encontrar diferentes momentos en el desarrollo de un repertorio docente que posibilite a los 
profesores una ensenanza de calidad. No deben entenderse las etapas como momentos fijos por los cuales todos los 
profesores estan obligados a pasar y en los mismos momentos, sino como indicadores flexibles (Richardson y Placier, 
2001 ). 

En la misma linea, Leithwood (1992) hizo una propuesta que vinculaba tres lineas de investigacion que hablan venido 
existiendo de manera independiente. Por una parte, las teorias sobre el desarrollo moral y psicologico de las personas, el 
desarrollo en la adquisicion de competencia profesional, y el desarrollo en el ciclo de la carrera. Como se observa en la 
Figura N® 2, el desarrollo de la competencia profesional es un proceso en el que se produce una evolucion, al igual que 
ocurre en las diferentes fases del desarrollo personal. Para Leithwood, la primera etapa supondria el Desarrollo de 
destrezas de supervivencia, en la que el profesor llega a dominar destrezas de gestion de clase, a utilizar diferentes 
modelos de ensenanza, aunque los profesores en esta etapa sean incapaces de reflexionar sobre la eleccion de un modelo u 
otro. A nivel de desarrollo personal, esta fase se corresponderia con la fase de autoproteccion, y concretismo cognitivo. 

La segunda fase seria la de Competencia en las destrezas basicas de ensenanza, e implica que el profesor posee destrezas 
de gestion de clase bien desarrolladas; destrezas en el uso de distintos modelos de ensenanza y de evaluacion formativa de 
los alumnos. El tercer nivel se denomina de Desarrollo de flexibilidad instruccional, y es aquel en el que el profesor tiene 
automatizadas las destrezas de gestion de clase; es consciente de la necesidad de conocer y manejar otros modelos de 
ensenanza, de forma que la eleccion de un modelo de ensenanza se realiza a partir de analizar el interes para los alumnos. 
Esta es una etapa que se relaciona con una mayor consciencia moral de los profesores, asi como por una implicacion en 
nuevas actividades que le permitan un mayor desarrollo profesional. En la cuarta fase, los profesores adquieren 
Competencia profesional de forma amplia y reflexiva, tomando la gestion de clase integrada en un programa, y no tratada 
de forma independiente, asi como se posee ya un dominio experto en la aplicacion de un amplio repertorio de modelos de 
ensenanza. La evaluacion de los alumnos se lleva a cabo de forma formativa y sumativa, utilizando una variedad de 
tecnicas. 

Si las anteriores etapas representaban un proceso de adquisicion de competencia profesional a nivel de ensenanza, en las 
siguientes etapas el componente didactico deja su lugar al de asesoramiento. Leithwood identifica la etapa que denomina 
Contribucion al desarrollo didactico de los companeros, en la cual el profesor puede desempenar actividades de 
asesoramiento, mentorazgo, supervision de companeros que se encuentren en etapas previas. La ultima etapa del desarrollo 
de la Competencia profesional la denomina Leithwood como de Participacion en decisiones educativas de alto nivel en el 
sistema educativo. 
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Figura N° 3. Modelo de integraci6n del Desarrollo Psicol6gico, Ciclos Vitales y Desarrollo de la Competencia 

profesional 

Una de las etapas claramente caracterizadas en las diferentes investigaciones tiene que ver con los primeros anos de 
ensenanza, denominado periodo de iniciacion o insercion profesional. Este ha sido un periodo que ha recibido mucha 
atencion por parte de los investigadores. 



La inserci6n profesional en la ensenanza, como hemos dicho, es el periodo de tiempo que abarca los primeros anos, en 
los cuales los profesores han de realizar la transicion desde estudiantes a profesores. Es un periodo de tensiones y 
aprendizajes intensivos en contextos generalmente desconocidos y durante el cual los profesores principiantes deben 
adquirir conocimiento profesional ademas de conseguir mantener un cierto equilibrio personal. Es este el concepto de 
induccion que asume Vonk, autor holandes con una decada de investigaciones centradas en este ambito.- ''definimos la 
induccion como la transicion desde profesor en formacion hasta llegar a ser un profesional autonomo. .. La induccion se 
puede entender mejor como una parte de un continuo en el proceso de desarrollo profesional de los 
profesores'* ( 1 996: 1 1 5). 



Los profesores principiantes tienen, segun Feiman (2001) dos tareas que cumplir: deben ensefiar y deben aprender a 
ensenar. Independientemente de la calidad del programa de formacion inicial que hayan cursado, hay algunas cosas que 
solo se aprenden en la practica y ello repercute en que este primer ano sea un afio de supervivencia, descubrimiento, 
adaptacion, aprendizaje y transicion. Las principales tareas con que se enfrentan los profesores principiantes son: adquirir 
conocimientos sobre los estudiantes, el curriculo y el contexto escolar; disenar adecuadamente el curriculo y la ensenanza; 
comenzar a desarrollar un repertorio docente que les permita sobre vivir como profesor; crear una comunidad de 
aprendizaje en el aula, y continuar desarrollando una identidad profesional. Y el problema es que esto deben hacerlo 
cargados con las mismas responsabilidades que los profesores mas experimentados (Marcelo, 1999). 



Ray Bolam, autor ingles, tambien dedicado durante anos al estudio del periodo de insercion, lo define como "el proceso de 
apoyo y formacion que de forma creciente se ve necesario para el exito en el primer ano de ensenanza. Los politicos 
tienden a interesarse mas por la induccion cuando existen problemas de reclutamiento de profesores, viendolo como el 
medio que mejora la retencion, consiguiendo animar a los profesores principiantes para que permanezcan en la docencia. 
Los profesionales tienden a estar interesados en la induccion al considerarla un periodo puente entre la formacion inicial 
y la permanente, y por tanto un periodo que sirve de fundamento para del desarrollo profesional continuo" (1995:613) 

El periodo de insercion profesional se configura como un momento importante en la trayectoria del futuro profesor. Un 
periodo importante porque los profesores deben realizar la transicion de estudiantes a profesores, por ello surgen dudas, 
tensiones, debiendo adquirir un adecuado conocimiento y competencia profesional en un breve periodo de tiempo . En este 
primer ano los profesores son principiantes, y en muchos casos, incluso en su segundo y tercer ano pueden todavia estar 
luchando para establecer su propia identidad personal y profesional (Esteve, 1997). 

Los cambios que sufre el profesor al pasar de estudiante a profesor principiante y de este a experto o experimentado, se 
suceden a traves de una serie de estadios, "que normalmente se ven como irreversibles de alguna manera" {Burden^ 

1990:31 1). Lo que conduce al cambio son, fundamentalmente, factores madurativos dentro del individuo y factores 
interactivos entre las caracteristicas personales y la estimulacion que reciben del entomo. 

Fue Simon Veenman (1 984) quien ha popularizado el concepto de "choque con la realidad" para referirse a la situacion por 
la que atraviesan muchos ensenantes en su primer ano de docencia. Segun este autor holandes, el primer ano se caracteriza 
por ser, en general, un proceso de intenso aprendizaje -del tipo ensayo-error en la mayoria de los casos-, y caracterizado 
por un principio de supervivencia, y por un predominio del valor de lo practico. Los programas de iniciacion tratan de 
establecer estrategias para reducir o reconducir el denominado "choque con la realidad". Los profesores principiantes se 
encuentran con ciertos problemas especificos de su estatus profesional. Valli (1992) plantea que son la imitacion acritica 
de conductas observadas en otros profesores; el aislamiento de sus companeros; la dificultad para transferir el 
conocimiento adquirido en su etapa de formacion, y el desarrollo de una concepcion tecnica de la ensenanza, los problemas 
que mas amenazan a los profesores principiantes. 

El proceso que siguen los profesores para aprender a ensenar, esto es, para adquirir competencia y habilidad como 
docentes ha sido explicado desde diferentes perspectivas teoricas. En un caso se hace hincapie en las preocupaciones de 
los profesores como indicadores de diferentes etapas de desarrollo profesional. En otro caso se concibe al profesor desde 
un punto de vista fundamentalmente cognitivo y al aprender a ensenar como un proceso de madurez intelectual. Existe un 
ultimo marco para el andlisis del proceso de iniciacion que hace mayor hincapie en los elementos sociales y culturales de 
la profesion docente y en su asuncion por parte del profesor principiante. 

Desde este ultimo enfoque, se estudia el periodo de iniciacion como un proceso mediante el cual los nuevos profesores 
aprenden e interiorizan las normas, valores, conductas, etc. que caracterizan a la cultura escolar en la que se integran. Se 
entiende que la socializacion "es el proceso mediante el cual un individuo adquiere el conocimiento y las destrezas 
sociales necesarias para asumir un rol en la organizacion" {Van Maanen y Schein, 1979:21 1). 



El periodo de iniciacion a la ensenanza representa el ritual que ha de permitir transmitir la cultura docente al profesor 
principiante (los conocimientos, modelos, valores y simbolos de la profesion), la integracion de la cultura en la 
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personalidad del propio profesor, asi como la adaptacion del profesor principiante al entomo social en que se desarrolla su 
actividad docente. Esta adaptacion puede ser facil cuando el entomo socio-cultural coincide con las caracteristicas del 
profesor principiante. Sin embargo, este proceso puede ser mas dramatico cuando el profesor principiante debe integrarse 
en culturas que les son desconocidas hasta el momento de empezar a ensenar. Es el caso de profesores principiantes 
ensenando en aulas multiculturales, de las cuales ban tenido escasa informacion hasta el momento de ser destinados a una 
de ellas. 

6.3. Los programas de formaci6n 

Una tercera tematica de investigacion que hemos escogido abarca los diferentes componentes del curriculo de la formacion 
del profesorado. En este amplio tema tendremos que diferenciar entre los programas dirigidos a los profesores en 
formacion, a los profesores principiantes y los programas de formacion tendentes al desarrollo profesional de los 
profesores. 



Los programas de formacion inicial 

iQuc deben saber los profesores? Algunas respuestas se han venido dando a partir de la investigacion sobre el 
conocimiento de los profesores a la que anteriormente hemos hecho referencia. Pero a pesar de ellas, hay escasez de 
investigaciones que hayan tornado como objeto de estudio el programa de formacion de profesorado en su totalidad. Hay si 
muchas investigaciones sobre elementos especificos de los programas, especialmente en relacion con las practicas de 
ensenanza. Y en estas investigaciones generalmente queda ausente la descripcion y analisis de los componentes de 
ensenanza practica en las instituciones de formacion, asi como la tarea de los formadores. Y resulta casi una verdad 
asumida la dificultad de los programas de formacion para modificar las creencias que los estudiantes traen consigo cuando 
se incorporan a un programa de formacion (Richardson y Placier, 2001). Feiman, que ha investigado a lo largo de toda su 
vida sobre la formacion de los profesores, concluye su revision con las siguientes palabras: "Los programas tradicionales 
de formacion inicial del profesorado y de desarrollo profesional no estdn disenados para promover aprendizajes 
complejos ni en los profesores ni en los alumnos. El tipico programa de formacion inicial representa una intervencion 
muy debil comparada con la influencia que los profesores en formacion han tenido en su etapa escolar, asi como de las 
experiencias de practicas " (200 1:1014). 

Darling- Hammond y MacDonald (2000) han analizado siete programas de formacion del profesorado norteamericanos 
considerados como programas eficaces. El analisis comparado de sus caracteristicas les lleva a concretar una serie de 
elementos que los caracterizan: 

• Coherencia conceptual: La coherencia conceptual es el elemento mas importante en un programa de formacion 
inicial del profesorado. Proporciona una vision orientadora acerca del tipo de profesor que se esta formando, una 
vision de aprendizaje, del papel del profesor y de la escuela, e incluye los valores y creencias que despues se ven 
en el curriculo y en las oportunidades de aprendizaje para los alumnos en practicas. 

• Practicas de ensenanza integradas y con un proposito: La coherencia en el segmento de practicas de ensenanza es 
un elemento fundamental: observaciones, orientacion, practica guiada, aplicacion de conocimiento e indagacion 
son elementos importantes. En los programas que estudian Darling-Hammond y MacDonald, las practicas estan 
organizadas, para promover reflexion y un aprendizaje teorico, utilizan diarios, tareas, seminarios semanales, etc. 
Se cuida la localizacion multiple de los alumnos en practicas. Igualmente, todos los alumnos en practicas trabajan 
con profesores supervisores que han completado cursos especiales ofrecidos por la Universidad. Incluyen 
mentoring y evaluacion. 

• Atencion a los profesores como sujetos que aprenden: La meta del programa de formacion del profesorado es que 
los profesores aprendan. Los profesores se esfuerzan porque los alumnos sigan aprendiendo mediante el apoyo a 
nuevas ideas de los alumnos, compartiendo ideas y conocimientos. Un reconocimiento del alumno como sujeto 
que aprende comienza con el reconocimiento de que los profesores en formacion llegan con creencias e imagenes 
que deben ser transformadas. 

Una de las criticas a los programas de formacion del profesorado es su escasa adaptacion a los cambios que se producen, y 
que hemos analizado en la primera parte de este articulo, en relacion con la diversidad de los estudiantes en las aulas. De 
man era grafica, Ladson-Billing afirmaba que "se sigue formando a los profesores para ensenar en escuelas ideales con 
ninos blancos, monolingiies, de clase media yde familias con dos padres" {\99%:%1). A esta critica se unen Grant y 
Wieczorek (2000) al destacar la ausencia de un analisis critico sobre los aspectos y condicionamientos sociales del 
conocimiento, que pueden encontrarse en temas como la raza, clase social, genero y poder, asi como los elementos 
historicos y politicos ligados a la produccion de conocimiento. Al respecto, ya en 1992, Zeichner alertaba sobre la 
necesidad de atender a la diversidad desde dentro de los programas de formacion inicial, cuyos elementos podrian ser los 
siguientes: 
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ELEMENTOS CLAVES DE UN PROGRAMA DE FORMACION DE PROFESORES PARA LA 
DIVERSIDAD 

Se ayuda a los estudiantes a desarrollar un mayor conocimiento de su propia identidad etnica y cultural 
Se ayuda a los estudiantes a examinar sus propias creencias hacia otros grupos etnoculturales 
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Se ensena a los estudiantes la dinamica de prejuicios y racismo, y a como abordarla en educacidn 

Se ensefia a los estudiantes sobre la opresion economica y sobre las practicas educativas que contribuyen a la 
reproduccion de las desigualdades sociales 

El curriculum de la formacion del profesorado aborda la historia y la contribucion de los diferentes grupos 
etnoculturales 

Se proporciona informacion a los estudiantes sobre las caracteristicas y estilos de aprendizaje de diferentes grupos e 
individuos, y se les ensena sobre las limitaciones de esta informacion 

El programa de formacidn presta mucha atencion a la investigacion sobre el conocimiento sociocultural que aborda 
las relaciones entre lenguaje, cultura y aprendizaje 

Se ensena a los estudiantes diferentes procedimientos por los cuales pueden obtener informacion acerca de las 
comunidades presentes en la clase 

Se ensena a los estudiantes como valorar las relaciones entre los metodos que se utilizan en la clase, los estilos de 
aprendizaje, y los estilos de interaccion en los hogares y comunidades de los estudiantes 

Se ensena a los estudiantes a utilizar una variedad de estrategias de enseflanza y de evaluacion que sean sensibles a 
las variaciones culturales y linguisticas, asi como a adaptar la enseftanza y la evaluacion a los recursos culturales que 
los estudiantes aportan a la escuela 

A los estudiantes se les presentan ejemplos de ensenanza eficaz con estudiantes de minorias etnicas y linguistica 

Los estudiantes realizan experiencias practicas con adultos y/o ninos de otros grupos etnoculturales, siguiendo una 
reflexibn guiada 

Los estudiantes realizan unas practicas de ensenanza en escuelas a las que asisten estudiantes procedentes de 
minorias etnicas y linguisticas 

Los estudiantes viven y ensenan en una comunidad minoritaria (inmersion) 

La ensenanza se desarrolla en una situacion de grupo que propicia el apoyo tanto intelectual como social 



Quizas el componente curricular mas estudiado en los programas de formacion inicial de profesores sean las practicas de 
ensenanza. Las practicas de ensenanza se asumen como uno de los componentes curriculares mas importantes de la 
formacion del profesorado por varios motivos: les permite a los profesores en formacion tomar contacto con situaciones 
reales de ensenanza; es una situacion de aprendizaje mediante la experiencia; suponen un alto grado de implicacion 
emocional; producen crecimiento; las metas se determinan intemamente en lugar de extemamente como es habitual 
(McIntyre, Byrd y Foxx, 1996). Durante ellas se espera que los alumnos apliquen sus conocimientos teoricos en las 
practicas de ensenanza. Se ha analizado el efecto que las practicas tienen en los profesores en formacion (Zeichner, 1999), 
Ilegandose a la conclusion de que el incremento de practicas por si solas sin analisis ni reflexion no conduce a un 
observable desarrollo profesional. Por otra parte el papel de los profesores tutores que reciben a sus aulas a los alumnos en 
practicas sigue estando poco investigado. Se sabe de su influencia en algunos alumnos pero no mucho mas. 



Los programas de insercion profesional 



Ya hemos descrito anteriormente que una linea de investigacion que se ha llevado a cabo ha tenido que ver con el analisis 
de los problemas de los profesores principiantes durante su periodo de insercion profesional. Como consecuencia de estos 
problemas, y de detectar la importancia del periodo de insercion en el subsiguiente desarrollo profesional de los docentes 
es por lo que se han venido desarrollando programas de insercion. Algunos programas de iniciacion incluyen, entre sus 
actividades, el asesoramiento de los profesores principiantes por medio de otros profesores, que pueden ser companeros o 
bien "mentores" (Vaillant y Marcelo, 2001). 



A traves de la revision de la literatura, se comprueba que los mentores, en general, son profesores de universidad, 
supervisores, directores de escuela, pero, en muy pocas ocasiones, companeros de los profesores principiantes. 
Independientemente de ello, es precise destacar la figura del mentor como elemento importante de los programas de 
iniciacibn, y senala que sus caracteristicas han de ser las siguientes: profesor permanente, con experiencia docente, con 
habilidad en la gestibn de clase, disciplina, comunicacibn con los compafieros, con conocimiento del contenido, con 
iniciativa para planiflcar y organizar, con cualidades personales (flexibilidad, paciencia, sensibilidad), etc. La tarea que se 
asigna al "mentor” es la de asesorar didactica y personalmente al profesor principiante, de forma que se constituye en un 
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elemento de apoyo. En algunos casos se pueden desarrollar ciclos de supervision clinica (planificacion-observacion- 
analisis de la ensenanza), o bien entrevistas abiertas. 

En otro trabajo ya he tenido ocasion de repasar tanto la investigacion sobre profesores principiantes como estos programas 
(Marcelo, 1999), por lo que dirigimos al lector a este trabajo que puede consultarse en Internet. Otros investigadores que 
han revisado el periodo de insercion, asi como los programas que se han en marcha, han si do Wideen, Mayer-Smith y 
Moon (1 998), y concluyen ha habido mucha investigacion sobre el primer ano de ensenanza, confirmandose la vision 
ampliamente extendida de que este ano supone un choque cultural para los profesores principiantes, especialmente para los 
que estan peor pre parados. Un aspecto negativo que senalan los autores es el siguiente: "encontramos una poblacion 
homogenea de profesores principiantes intentando aprender a ensenar a una poblacion heterogenea de alumnos en las 
escuelas. Tambien encontramos muchos programas que tiene muy poco efecto sobre las creencias fuertemente asentadas 
acerca de la ensenanza que los profesores traen a sus programas de formacion. Tales programas frecuentemente parecen 
tener propositos cruzados con las experiencias que los profesores en formacion se encuentran durante sus prdcticas de 
ensenanza y su primer ano como docentes." (159). Estos programas solo cumplen una funcion burocratica pero no hace 
por crear una identidad profesional en los docentes. 

Los programas para el desarrollo profesional 



En la primera parte de este articulo hemos analizado los procesos de aprendizaje en la sociedad informacional. Estudiamos 
el interes que estan generando las ideas acerca del aprendizaje a lo largo de toda la vida, asi como del aprendizaje informal, 
informal. El modelo de aprendiz, deciamos citando a Himanen, era el de un sujeto con autonomia y capacidad de indagar, 
de aprender compartiendo y ensenando a otros. Un modelo de aprendizaje contextualizado y distribuido. 



^Cuales son las conclusiones que podemos extraer de las revisiones acerca de los programas de desarrollo profesional? 
Quizas hay un resultado evidente y es que el paradigma tradicional mediante el cual la formacion continua se organiza en 
tomo a unidades discretas de conocimientos o habilidades, impartidas por expertos, en lugares alejados de las escuelas, con 
una duracion limitada, con escaso seguimiento y aplicacion practica no tienen ninguna posibilidad de cambiar ni las 
creencias ni las practicas docentes de los ensenantes. Joyce encontro que las posibilidades de implantacidn de cambios 
siguiendos este modelo eran del 15% (Richardson y Placier, 2001). 

Analizar los modelos de desarrollo profesional significa hacer referencia a los modelos de cambio y reforma que a lo largo 
de los ultimos cincuenta anos se han venido asumiendo. Y, como se podra comprender, a estas alturas del articulo, no 
puede ser nuestra intencion entrar a comentar lo que ya de forma magistral otros autores han realizado. Especialmente 
importante ha sido el International Handbook of Educational Change, editado por Hargreves, Lieberman, Fullan y 
Hopkins (1998). Y de especial interes resulta la aportacidn de Mat Miles sobre 40 anos de cambio (Miles, 1 998). 

^Como pues, podemos desarrollar programas que afecten a los profesores mejorando su practica docente? Feiman (2001) 
concretaba una idea que ya ha venido tomando cuerpo por parte de otros investigadores y formadores: el hecho de que lo 
que los profesores deben aprender han de hacerlo en la practica. Los profesores necesitan aprender como aprender de la 
practica, puesto que la ensenanza requiere improvisacion, conjetura, experimentacion y valoracion. Pero aprender en la 
practica no es una proceso que se de naturalmente. Ball y Cohen (1999) establecen tres condiciones para poder esperar 
algun aprendizaje a partir de la experiencia practica: a) los profesores tienen que aprender a adecuar sus conocimiento a 
cada situacion, ello significa indagar acerca de lo que los estudiantes hacen y piensan y como comprenden lo que se les ha 
ensenado; b) los profesores deben aprender a utilizar su conocimiento para mejorar su practica, y c) los profesores 
necesitan aprender como enmarcar, guiar y revisar las tareas de los alumnos. Desde su punto de vista, "/a clave de nuestra 
respuesta es que estar centrado en la practica no necesariamente implica situaciones en las aulas en tiempo real" (14). Es 
decir, para aprender, los profesores necesitan utilizar ejemplos practicos, material es como casos escritos, casos 
multimedia, observaciones de ensenanza, diarios de profesores y ejemplos de tareas de los alumnos. Estos materiales 
podrian permitirque los profesores indagaran acerca de la practica, analizaran la ensenanza. 



Otros investigadores han intentado sintetizar las caracteristicas de los programas de formacion permanente que se han 
mostrado eficaces. Abdal-Haqq (1995) los caracteriza como: progresivos; incluyen formacion, practica y retroaccion, 
oportunidades para reflexion individual e indagacion grupal sobre la practica, y seguimiento; estan basados en la escuela e 
implican al trabajo del profesor; son colaborativos, proporcionando oportunidades para que los profesores interactuen unos 
con otros; se centran en el aprendizaje de los alumnos, el cual deberia servir para evaluar la eficacia de la formacion; 
anima y apoya iniciativas de profesores en las escuelas; estan basados en el conocimiento base para ensenar; incorpora 
enfoques constructivistas del aprendizaje y la ensenanza; reconoce a los profesores como profesionaies su adultos que 
aprenden; y proporciona un adecuado tiempo y seguimiento (en Wilson y Berne, 1999). 




Parece que existe coincidencia en la necesidad de entender que los profesores tienen una gran responsabilidad en dar 
respuesta a las nuevas demandas sociales generadas por los cambios a los que hemos aludido en la primera parte de este 
articulo. Se espera de los profesores la necesidad de responder a los cambios en el conocimiento y el aprendizaje; la 
necesidad de desarrollar habilidad en el uso de nuevas tecnologias; la necesidad de redirigir sus enfoques pedagogicos 
hacia una ensenanza y aprendizaje mas personal izados; la necesidad de acomodarse a la presencia en clase de alumnos de 
diferentes edades y etapas de desarrollo cognitivo; la necesidad de desarrollar una orientacion diferente frente a la 
autoridad y estilos de gestion de aula; la necesidad de entrar en contacto con otras agencias e instituciones que promueven 
aprendizaje formal o informal (Chapman, 1996). Y para hacer frente a estas necesidades es preciso buscar nuevas formulas 
mas acordes con un aprendizaje para el cambio. 
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Una de estas iniciativas han sido las redes. La formacion del profesorado ha abierto de forma tradicional espacios para el 
aprendizaje entre iguales, para la comunicacion entre docentes al margen generalmente de las ofertas oficiales. Aunque 
minoritarios en sus origenes han existido movimientos que han reivindicado una formacion participativa, democratica, 
horizontal y profesional. Muchos son los ejemplos que podemos poner de iniciativas en las que los protagonistas de la 
formacion son los propios profesores. El termino red se entiende como una mall a de personas conectadas por enlaces en 
tomo a los cuales fluyen cosas como objetos, trabajo, afectos, evaluacion, conocimiento, prescripciones, influencia y poder, 
y en las que la mayoria de los participantes estan conectados unos con otros. Las redes enlazan a diferentes personas con 
diferentes propositos utilizando una variedad de formulas (Marcelo, 2001b). 

Se han venido aportando diferentes definiciones de lo que se entiende por una red. Miles la definia como "w/r conjunto de 
nodos o punt os conectados por tineas o enlaces. Existe a menudo la implicacion de que varias cosas (mensajes, objetos, 
energia, etc.) viajan a lo largo de tineas, que asi sirven como canal... En las redes sociales, los nodos son las personas, 
grupos u organizaciones. Las cosas que viajan entre los nodos son socialmente relevantes... objetos, trabajos, afectos, 
evaluacion, conocimiento, prescripcion/opinion, influencia y poder. De esta forma, una red es un conjunto conectado de 
actores sociales intercambiando materiales socialmente relevantes” {C\X. En Clark, 1988:34). De la misma forma, 
Lieberman y Grolnick, entiende que "Las redes const ituyen una forma de imp Hear a los profesores en la direccion de su 
propio aprendizaje: les permite superar las limitaciones de sus roles institucionales,jerarquiasylocalizacidn geogrdfica; 
y les anima a trabajar juntos con muchas gentes diferentes. Los participantes tienen la oportunidad de crecery 
desarrollarse en una comunidad profesional que se centra en su propio desarrollo, proporcionando formas de aprendizaje 
que tienen mas que ver con las experiencias profesionales vividas" (1996:8-9). Por ultimo, Moonen y Voogt definen una 
red como un como "un grupo de profesores de diferentes escuelas que cooperan durante un largo periodo de tiempo en la 
implantacion de un proceso que conlleva una cierta innovacion en educacion" (1998: 103). 



El florecimiento de las redes entre profesores tiene mucho que ver con la aversion de los profesores hacia las actividades 
tradicionales de formacibn permanente. La popularidad de las redes sugiere que los profesores se alejan de las actividades 
convencionales de formacion -o asisten a ellas solo cuando se les requiere- no debido a una falta de interes en el desarrollo 
profesional si no debido a que el formato de la formacion permanente no responde a sus necesidades (Lieberman y 
McLaughlin, 1996). Pero las Redes solo van a representar un espacio altemativo a la formacion tradicional en la medida en 
que promuevan el aprendizaje y la reflexion sobre la experiencia cotidiana. Day recientemente lo decia con la mayor 
claridad: ”Por definicion, las redes suponen el reconocimiento de que aprender solo de las propias experiencias limita el 
desarrollo, y que es probable que los profesores se comprometan con un tipo aprendizaje que tenga significado para ellos. 
En esencia, por tanto, las redes proporcionan estructuras organizativas que permiten a los grupos de profesores reunirse 
para hablar de su trabajo, aprender de otros, y conducir temas curriculares y estructurales” (Day, 1 999: 177). 



Las redes configuran espacios en los que el aprendizaje esta distribuido, donde es posible construir colaborativamente y 
donde los profesores son los protagonistas y responsables de la formacion. Las Nuevas Tecnologias de la Informacion y 
Comunicacibn, como he tenido ocasibn de desarrollar en un libro de reciente aparicibn (Marcelo, 2002), ayudan y facilitan 
la creacibn de estas redes que mediante el correo electrbnico, el chat o los foros comparten y crean conocimiento. (Nota 2) 

Conclusiones 




Memos hecho un recorrido por los nuevos y los viejos temas que han venido ocupando la atencibn de los investigadores 
que se preguntado sobre los procesos de aprender a ensenar. No ha sido una revisibn exhaustiva porque los temas son 
muchos y el espacio escaso. Podemos tener la sensacibn de que vamos avanzando en el conocimiento sobre el aprender a 
ensenar, pero si miramos a nuestro alrededor vemos que muchas de las practicas mas tradicionales permanecen. Me llamb 
mucho la atencibn un articulo de Labaree (1998) en el que, despues de analizar el tipo de conocimiento sobre el que 
trabajamos los investigadores educativos, concluia con una frase que me permito reproducir. Decia que ""Un problema que 
el conocimiento educativo plantea a aquellos que buscan producirlo es que a menudo les deja con la sensacion de estar 
perpetuamente luchando por avanzar hacia ninguna parte. Si Sisifo fuera universitario, su campo seria la educacion. Al 
final de una larga y distinguida carrera, muchos investigadores en edad de jubilacion suelen encontrar que aun se 
encuentran trabajando en los mismos problemas que abordaban al comienzo de su carrera” (9). 

No es que uno se encuentre ya al final de su carrera, pero si que cuenta con el tiempo suficiente como para comprobar que 
muchos temas de investigacibn aparecen y desaparecen, sirven para marcar agendas investigadoras pero en pocas ocasiones 
los resultados (o los procesos) de investigacibn consiguen mantenerse en el tiempo para poder llegar a esponjar la cultura 
tradicional, es decir, lo que se espera de la formacibn, esa "sabiduria recibida” de que hablaba Kennedy (1998). Y por lo 
que vamos viendo en la investigacibn sobre la formacibn del profesorado, como nos recordaban Joyce y Calhoun (1998), 
puede que nos este pasando como en el chiste del borracho que buscaba la Have bajo la luz de la farola, y no en el sitio 
donde se le habia caido, con el pretexto de que debajo de la farola habia mas luz. ^Hemos buscado en los lugares 
adecuados? Si la formacibn del profesorado necesita de un replanteamiento de las creencias y concepciones de los 
aspirantes a profesor ^hemos analizado cbmo esto se lleva a cabo en las aulas de formacibn? Si Lee Shulman en 1986 
encontrb un "missing paradigm", una inexistente investigacibn en relacibn con el contenido que se ensena, hoy dia 
podemos decir que en la formacibn del profesorado hay un componente, no circunstancial ni trivial que ha sido 
escasamente investigado. Curiosamente, aquel que queda bajo la responsabilidad de los propios formadores: el propio 
proceso de aprender a ensenar en las instituciones universitarias de formacibn. La investigacibn sobre los formadores 
universitarios que imparten su docencia en la formacibn inicial, como atentamente nos marcaban Wideen, Mayer-Smith y 
Moon, es practicamente inexistente. Y lo mismo ocurre con investigaciones que adopten una visibn sistemica o ecolbgica y 
comprendan multiples elementos de los programas format! vos para conocer el efecto que tienen en los profesores en 
formacibn. 
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Los cambios que se estan produciendo en nuestra sociedad, y a los que dedicamos la primera parte de este arti'culo, nos 
estan demandando un giro tanto en los programas de formacion como en la investigacion sobre estos programas. Y para 
ello hace falta un esfuerzo y compromiso compartido hacia una direccion que fomente el aprendizaje, la innovacion, la 
flexibilidad, la autonomia, el trabajo en red y el aprendizaje colaborativo. Y quizas lo que hoy dia nos falte, y en esto 
coincido con Cochran-Smith, es una teoria de la formacion que apunte la direccion del cambio y ayude a no perderse en 
discursos vacios o en modas pasajeras. Una teoria que reconozca que la formacion del profesorado y la ensenanza son 
"actividades politicas e intelectuales a la vez que prdcticas, que ocurren en un contextos historico, economico y social 
complejo. Tal teoria, basada en los datos de la investigacion y de la prdctica, debe tener el potencial de guiar, sugerir 
criticas, y mds importante, sugerir formas de implementar, comprender e investigar la formacion del profesorado para el 
cambio (1998:918). Los elementos de que disponemos hoy dia nos permiten identificar algunas lineas maestras de 

esa teoria, pero aun nos quedan por resolver muchas cuestiones de investigacion que nos ayuden a construir e implementar 
buenos programas de formacion que nos permitan dotamos de buenos profesores que a su vez consigan hacer realidad uno 
de los derechos mas sagrados: el derecho de aprender. 

Notes 

1. Recomendamos la pagina de internet de F. Pajares: http://www.emory.edu/EDUCATION/mfp/ 

2. Para mas informacion sobre E-Leaming recomendamos la direccion: http://www.webformacion.net 
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Abstract 

In October, 2001, the Baltimore-based Abell Foundation issued a report purporting to prove that 
there is ”no credible research that supports the use of teacher certification as a regulatory barrier 
to teaching” and urging the discontinuation of certification in Maryland. The report argued that 
large inequities in access to certified teachers for poor and minority students are not a problem 
because research linking teacher education to student achievement is flawed. In July, 2002, the 
U.S. Secretary of Education cited the Abell Foundation paper in his Annual Report on Teacher 
Quality as the sole source for concluding that teacher education does not contribute to teacher 
effectiveness. The Secretary's report then recommended that requirements for education 
coursework be eliminated from certification standards, and attendance at schools of education 
and student teaching be made optional. This article documents the many inaccuracies in the Abell 
Foundation paper and describes the actual findings of many of the studies it purports to review, 
as well as the findings of other studies it ignores. It details misrepresentations of a number of 
studies, including inaccurate statements about their methods and findings, false claims about 
their authors' views, and distortions of their data and conclusions. The article addresses 
methodological issues regarding the validity and interpretation of research. Finally, the article 
presents data challenging the Abell Foundation’s unfounded claims that uncertified teachers are 
as effective as certified teachers, that teacher education makes no difference to teacher 
effectiveness, that verbal ability is the most important determinant of teaching effectiveness, that 
private schools staffed by uncertified teachers are more effective than public schools, and that 
untrained teachers are more qualified than prepared teachers. It concludes with a discussion of 
the policy issues that need to be addressed if all students are to be provided with highly qualified 
teachers. 



o 

ERIC 



In October, 2001, the Baltimore-based Abell Foundation issued a report purporting to prove that there is "no credible 
research that supports the use of teacher certification as a regulatory barrier to teaching" (Walsh, 2001, p. 5). (Note 2) 
The Abell Foundation paper argued against Maryland's efforts to strengthen teacher preparation requirements and 
defended the continuation of a local short-term alternative route into teaching that had come under criticism. 
Suggesting that "educators, policymakers, the media, and the public mistakenly equate teacher quality with teacher 
certification" (p. 1), Kate Walsh, the author of the paper, complained that efforts to improve education for poor and 
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minority children in Baltimore by the state and local superintendents of schools and by local advocacy organizations 
foolishly sought to secure more fully certified teachers for their schools. She cited as wrong-headed newspaper 
articles raising concerns, for example, that: "Least prepared teachers are at worst city schools: One-third lack basic 
credentials for certification," (p. 1). Calling misguided the efforts of a Baltimore community group that released a 
study which "bemoaned the fact that more uncertified teachers were teaching in the city's high-poverty, predominantly 
African-American schools than the city's whiter, more affluent schools" (p. 2), the paper sought to demonstrate that 
these inequalities in access to certified teachers are not problematic if certification can be discounted as a determinant 
of achievement. 

The Abell Foundation proposed that Maryland should 1) "eliminate the coursework requirements for teacher 
certification" and require only a bachelor's degree and a passing score on an appropriate teacher's exam; 2) "report the 
average verbal ability score of teachers in each school district and of teacher candidates graduating from the State's 
schools of education;" and 3) "devolve its responsibility for teacher qualification and selection to its 24 public school 
districts," delegating all hiring authority to individual school principals (pp. vii-viii). 

Although these ideas might seem indefensible to those who are engaged in research regarding teacher preparation and 
recruitment, the U.S. Secretary of Education echoed these recommendations in his Annual Report on Teacher Quality 
(USDOE, 2002), a report on the national state of teacher quality required under the 1998 reauthorization of Title II of 
the Higher Education Act. In this report, the Secretary argued that teacher certification systems are "broken," 
imposing "burdensome requirements" for education coursework that make up "the bulk of current teacher certification 
regimes" (p. 8). The report argues that certification should be redefined to emphasize higher standards for verbal 
ability and content knowledge and to de-emphasize requirements for education coursework, making attendance at 
schools of education and student teaching optional and eliminating "other bureaucratic hurdles" (p. 19). 

The report suggests that its recommendations are based on "solid research." However, only one reference among the 
report's 44 footnotes is to a peer-reviewed journal article (which is misquoted in the report); most are to newspaper 
articles or to documents published by advocacy organizations, some of these known for their vigorous opposition to 
teacher education. (Note 3) For the recommendation that education preparation be eliminated or made optional, the 
Secretary's report relies exclusively on the Abell Foundation's paper. Though written as a local rejoinder to 
Maryland's efforts to strengthen teacher preparation and certification, it appears to have become a foundation for 
federal policy. 



This article includes the response I wrote to Walsh's paper (Note 4) when it was first issued, with some additions that 
respond to a reply she issued with Michael Podgursky (Note 5) and a briefer version of her report recently printed in 
Education Next, a magazine put out by the Hoover Institution (Walsh, 2002). 

In order to make a case for her agenda, Walsh attacks all research that has found relationships between teachers' 
preparation and their measured effectiveness, including students' achievement. She characterizes much of the 
education research as "flawed, sloppy, aged and sometimes academically dishonest" (p. 13), a characterization that 
more aptly describes her own paper, which consistently misrepresents the statements of researchers, the findings of 
studies, and the evidence base for her claims. She claims to have reviewed all of the studies ever cited by proponents 
of teacher education. In fact, a large number of the references in the paper and appendix are not directly on the topic 
of teacher education, and many studies of teacher education effects are not included in the report. Furthermore, her 
paper does not actually review most of the studies it mentions. An original report appendix listing studies shrank from 
175 in July, 2001 to fourteen in the version ofthe report released in October, 2001 selected according to no obvious 
criteria and omitting many of the most prominent studies on the topic. (Note 6) The "reviews" in a now separate 
appendix published on the foundation's website are generally not careful assessments of research methods or findings 
but a list of complaints and random observations — sometimes accurate but often not — about various aspects ofthe 
studies or how they have been cited by others. (A number of examples are included below.) 

All studies have limitations, and some are too problematic to be relied upon, including a number that Walsh relies 
upon for her own assertions. However, Walsh's paper, which is littered with inaccuracies, misstatements, and 
misrepresentations, sheds little light on the research or its implications for teacher education and certification. In what 
follows I discuss the inaccuracies in Walsh's account, the actual findings of many of the studies she purports to 
review, and the findings of other studies she chooses to ignore, as well as the implications of her proposals for 
teachers, their knowledge, and the students they teach. 



In the course of the paper, I review some of the studies that have found influences of teacher education and 
certification on student achievement at the levels of the individual teacher (e.g. Goldhaber & Brewer, 2000; Hawk, 
Coble, & Swanson, 1985; Monk, 1994); the school (Betts, Rueben, & Danenberg, 2000; Fetler, 1999); the school 
district (Ferguson, 1991; Strauss & Sawyer, 1986); and state (Darling-Hammond, 2000c). The convergence of 
findings in analyses using different units of analysis reinforces the strength of the inferences that might be drawn from 
any single study. 



What are the Arguments? 

The Abell Foundation report admits that teacher qualifications make a difference but it also tries to make a case that 
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"the backgrounds and attributes characterizing effective teachers are more likely to be found outside the domain of 
schools of education. The teacher attribute found consistently to be most related to raising student achievement is 
verbal ability.... usually measured by short vocabulary tests..." (p. v). Later in the report, Walsh suggests that subject 
matter knowledge may be an additional criterion for hiring secondary teachers, but not for elementary teachers. Walsh 
objects to the state requirements regarding content coursework in each of the core academic areas for elementary 
teachers, since many who want to enter through the alternative Resident Teacher program in Maryland have had 
trouble meeting these requirements. 



Walsh then tries to dismiss all studies that find evidence that knowledge about teaching also makes a different for 
teacher performance, or to claim that studies finding positive effects of teacher education or certification are either too 
old, too small, too highly aggregated, or dependent on evidence about teacher performance other than student 
achievement or are not really about certification after all, even if their authors say they are. She often does this by 
misrepresenting the studies' actual methods and findings, as I detail below. 

While there are legitimate concerns to be raised about various studies in the literature — on all sides of the question — 
this article does not shed much light on them. A thorough review of the quality and accurately portrayed findings of 
the several bodies of research that bear on this question would be a service to this field. Unfortunately, this 
document's inaccuracies and misinterpretations make it of little use in this regard. 



In what follows, I address five major issues regarding the Abell report and the research base on teaching and teacher 
education; 

1. Evidence Ignored. Evidence about student learning in reading and other areas documents the need for 
teachers to have professional knowledge that includes and extends beyond subject matter knowledge. The 
Abell Foundation report does not consider this evidence or answer the question of how teachers are to acquire 
this knowledge if they are not professionally prepared. 



2. Unfounded Claims. No evidence supports Walsh's claim that either verbal ability or subject matter knowledge 
alone makes teachers effective. She lacks supporting evidence — and fails to consider contradictory 
evidence — for her claims about the relative effectiveness of certified and uncertified teachers, the outcomes 
of teacher education, the primacy of verbal ability as the most important measure of teaching, the 
effectiveness of private and public schools and the preparation of their teachers, and the attributes of 
individuals who enter teaching without certification. 

3. Misrepresentations of Research. Walsh's claim that she has reviewed 1 00 to 200 studies cited in support of 
teacher education and found that "none of them holds up to scrutiny" is not true. In fact, she is unable to 
discount a number of important studies that support teacher education or certification. In addition, a large 
number of the studies relevant to the question of teacher education effects are not reviewed at all in Walsh's 
paper. Most of the studies she mentions do not concern teacher education or certification directly: at most 80 
of the nearly 200 studies listed in the study or appendix are focused on teacher education or certification. A 
number of those reviewed are badly misrepresented, including inaccurate statements about their methods and 
findings, false claims about their authors' views, and distortions of their data and conclusions. Many are not 
reviewed for their methods and findings, but are dismissed because of their sample size, age, dependent 
variable, or publication venue — unless Walsh likes one of the findings, in which case she uses the study, 
sometimes after already having dismissed it. Even the studies that Walsh says she reviewed are missing from 
the appendix of the report, where she refers readers for evidence. (Note 7) 

4. Methodological Issues and Double Standards in Using Research. Walsh misunderstands some fundamental 
research design issues, including the difference between experimental and correlational studies and the 
interpretation of research conducted at different levels of aggregation. In her effort to make the evidence base 
about teacher education disappear, Walsh eliminates from consideration studies that have been cited 
regarding the contributions of various measures of teacher qualifications to teacher effectiveness if they have 
small sample sizes, if they were published more than 20 years ago, or if they were published as dissertations, 
technical reports, or conference papers rather than in peer-reviewed journals. She also eliminates all studies 
that use measures of teacher effectiveness other than student achievement (e.g. supervisors' ratings of 
performance, researchers' obser\'ation-based measures of teacher practice). There are legitimate issues 
associated with the sample size, age, quality assurance, and measurement that warrant discussion (see below). 
However, as a blanket means of eliminating evidence from consideration, this strategy is problematic, as 
Walsh's frequent citations of studies that fail to meet her own criteria suggest. 

5. Illogical Policy Conclusions. While it is clear that teacher certification systems are not perfect and there are 
many weak teacher education programs, points that I have frequently made in my own research, it does not 
follow that the response to these problems should be to eliminate expectations for teachers to acquire the 
knowledge they need to teach students effectively. The more appropriate policy response is to improve the 
quality of teacher education — a process that has been underway with important results in a number of states, 
and one that rests on the processes of accreditation and certification that provide policymakers with levers for 
change and improvement. 
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Evidence Ignored 



While the Abell Foundation report claims that teachers do not need professional knowledge in order to teach, the field 
has been moving rapidly to codify the ways in which teaching knowledge makes a difference in student learning. For 
example, the National Reading Panel of the National Institute of Child Health and Human Development last year 
published a major review of carefully controlled research which found that children’s reading achievement is 
improved by systematic teaching of phonemic awareness, guided repeated oral reading, direct and indirect vocabulary 
instruction with careful attention to readers' needs, and a combination of reading comprehension techniques that 
include metacognitive strategies. 

The report notes that teacher education is critical to the success of reading instruction with respect to both instruction 
in phonemic awareness and more complex comprehension skills: 

Knowing that all phonics programs are not the same brings with it the implication that teachers must 
themselves be educated about how to evaluate different programs to determine which ones are based 
on strong evidence and how they can most effectively use these programs in their own classrooms. It 
is therefore important that teachers be provided with evidence-based preservice training and ongoing 
inservice training to select (or develop) and implement the most appropriate phonics instruction 
effectively, (p. 11) 

Teaching reading comprehension strategies to students at all grade levels is complex. Teachers not 
only must have a firm grasp of the content presented in the text, but also must have substantial 
knowledge of the strategies themselves, of which strategies are most effective for different students 
and types of content and of how best to teach and model strategy use.... (Data from the studies 
reviewed on teacher training) indicated clearly that in order for teachers to use strategies effectively, 
extensive formal instruction in reading comprehension is necessary, preferably beginning as early as 
pre-service (National Reading Panel, 2000, pp. 15-16). 



Studies have documented that professional training can be effective in providing teachers with the strategies that 
enable them to teach these complex comprehension skills, and teachers who receive such training significantly 
improve students' reading outcomes (e.g, Duffy, Roehler, Sivan et al., 1987; Duffy & Roehler, 1989, regarding 
explicit strategy instruction; Palincsar & Brown, 1989, regarding reciprocal teaching). 



Similar insights in our understanding of how to develop student proficiency in mathematics and science, and how to 
develop teachers' skills for doing so, have recently emerged. For example, recent analyses of the National Assessment 
of Educational Progress (NAEP) which control for student characteristics and a number of measures of school inputs 
have found that students whose teachers have majored in mathematics or mathematics education, who have had more 
pre- or in-service training in how to work with diverse student populations and more training in how to develop 
higher-order thinking skills, and who engage in more hands-on learning do better on the NAEP mathematics 
assessments. Similarly, students whose teachers have majored in science or science education and who have had more 
pre- or in-service training in how to develop laboratory skills and who engage in more hands-on learning do better on 
the NAEP science assessments (Weglinsky, 2000). (Note 8) 

A recent review commissioned by the Department of Education, which was carefully vetted by a panel of researchers, 
disagreed with the Abell Foundation's conclusions. This review, which analyzed 57 studies that met specific research 
criteria and were published after 1980 in peer-reviewed journals, concluded that the available evidence demonstrates a 
relationship between teacher education and teacher effectiveness (Wilson, Floden, & Ferrini-Mundy, 2001). The 
review shows that empirical relationships between teacher qualifications and student achievement have been found 
across studies using different units of analysis and different measures of preparation and in studies that employ 
controls for students' socioeconomic status and prior academic performance. 

It is ironic that just as the field is learning more about how to prepare teachers to teach children effectively, the Abell 
Foundation suggests that we truncate teacher education and end the certification policies that would encourage and 
enable teachers to acquire this knowledge — or at least that we do so for the children of the poor, who also attend 
school in districts with minimal resources for professional development. The unanswered question is, How are 
teachers to learn what is known about how to teach well if there are no expectations, incentives, or supports for them 
to do so? 



Unfounded Claims 

While ignoring these serious questions, Walsh makes a number of claims that are not supported either by the research 
she presents or by other evidence in the field. These include the following: 

• New teachers who are certified do not produce greater student gains than new teachers who are not certified. 



O 



There is little evidence that the content and skills taught in preservice education coursework is (sic) either 



retained or effective. 



• Verbal ability and subject matter alone are sufficient to produce effective teachers. 

• Private schools do not hire certified teachers and they are more effective than public schools. 

• Individuals with higher academic ability will be recruited to teaching if certification standards are eliminated. 
The Effectiveness of Certified and Uncertified Teachers 

For her proposition that "new teachers who are certified do not produce greater student gains than new teachers who 
are not certified," Walsh cites seven studies, none of which provides support for this proposition, and five of which 
actually provide evidence that contradicts her claim. Three of the studies (Bliss, 1992; Stoddart, 1992; Lutz & Hutton, 
1989) include no data on student achievement at all, although Walsh elsewhere dismisses all other studies that do not 
use student achievement data as the dependent variable. (In a reply to my response, Walsh and Podgursky (2001) note 
that these studies have been deleted in a newly printed version, along with some studies Walsh cited that were not 
peer reviewed, "so that the report ... does not appear to convey a double standard" (p. 15)). 

Six of the studies Walsh cites actually deal with alternatively certified rather than uncertified teachers — that is, 
teachers who had undertaken teacher education at the post-baccalaureate level in university- or school district-based 
programs that rearrange the way teacher education is delivered. The findings across the studies are mixed, but none of 
them shows that uncertified teachers do as well as certified teachers, and one of them shows that this is clearly not 
true. Several of the studies point instead to the value of teacher education: The more positive findings are found for 
the alternatives that provide more complete preparation. 

1. Bliss (1992) wrote about the Connecticut alternative certification program, a two-year training model which 
the author notes features "a significantly longer period of training than in any other alternate route program" 
in existence at that time (p. 52). This report does not examine uncertified teachers, nor does it meet Walsh's 
criteria for inclusion in a review of literature, because it includes no data about teacher effectiveness as 
gauged by student achievement measures. Bliss notes that most recruits reported their initial training to be 
helpful, and she briefly mentions results from another researcher's survey of recruits' supervisors which 
suggested mixed reviews of their performance: 33 percent of supervisors said that the alternate route teachers 
were weaker than others in classroom management (presumably, then, 67 percent said they were not weaker 
than others in this area), while 38 percent said they were stronger than others in teaching skills (and 62 
percent presumably said they were not stronger than others in this area). 

2. Stoddart (1992) reports on the subject matter qualifications and attrition rates of recruits to the Los Angeles 
Teacher Trainee Program, also a two-year training model. She found that content qualifications were 
comparable to those of traditionally trained recruits, except for math recruits, who had lower GPAs than 
traditionally trained mathematics teachers, and that attrition rates for those who entered were relatively low in 
the first two years but higher than national rates after 5 years. (Note 9) Results cited by Stoddart from other 
studies about the observed practices of these teachers in comparison with university-trained teachers 
produced mixed results: university-trained English teachers appeared more skillful than alternate route 
teachers, but the levels of skill appeared lower for mathematics teachers from both groups. 

3. Lutz and Hutton (1989) compared the demographic characteristics, attitudes, certification test scores, and 
opinions of Dallas Public Schools’ alternative certification (AC) recruits with other first year teachers in the 
district. Like the other studies noted above, this study did not examine student achievement gains of the 
recruits’ students. The program provides summer training to recruits and then places them in mentored 
internships during the school year while they are completing other coursework. The study found many 
similarities but some differences between AC recruits and other first year teachers, including significantly 
lower rates of expected long-term continuation in teaching for the AC recruits (40% vs. 72% for other first 
year teachers). They also examined supervisors' perceptions of recruits — a measure that Walsh argues should 
eliminate other studies from consideration. These were positive for the 54% of the pool (59 out of 1 10) 
defined as "successful" interns in the study — those who completed the intern year without dropping out 
(10%) or being held back for another year or more due to 'deficiencies' in various areas of performance 
(36%). The study also reported data from another evaluation of the program by the Texas Education Agency 
(Mitchell, 1987), which surveyed principals, finding that: 



The principals rated the [traditionally-prepared] beginning teachers as more knowledgeable 
than the AC interns on the eight program variables: reading, discipline management, 
classroom organization, planning, essential elements, ESL methodology, instructional 
techniques, and instructional models. The ratings of the AC interns on nine other areas of 
knowledge typically included in teacher preparation programs were slightly below average in 
seven areas compared with those of beginning teachers. It might therefore be assumed that 
pre-service teacher education programs are doing something right! (p. 250). 




In the paragraph cited above, Lutz and Hutton wax enthusiastic about preservice teacher education programs 
that seemed in these data to outperform the alternative route. Later they wax enthusiastic about the alternative 
route, given results from another survey of principals, most of whom felt that alternative credential candidates 
who eventually made through the program were comparable to other beginning teachers. At the end of the 
piece, they note that the high attrition rates and difficulty maintaining the program suggest the alternate route 
will not likely be a long-term solution to teacher supply problems. Although Walsh cites Lutz and Hutton’s 
enthusiastic feelings about the AC program, she does not accurately report the complete data from the study, 
including the low rates of successful program completion, the low rates of planned retention in teaching, and 
the mixed reviews of their performance. In her appendix, she includes this study with the following "review:" 
"Darling-Hammond ignores the unqualified authors' (sic) endorsement of the merits of alternative route to 
teaching...." One presumes that she means to reference the authors’ "unqualified endorsement" rather than to 
call the authors themselves unqualified. Yet as the above excerpts make clear, the study does not provide an 
unqualified endorsement of the program. 



Walsh repeats this mistake in the appendix when she critiques a review of alternate certification programs 
(Darling-Hammond, 1992). She states that, "Darling-Hammond cites the findings from many studies that 
looked at alternative programs; but she does not include findings that show alternatively trained teachers are 
at least as effective at raising academic achievement as those who graduate from traditional programs," (p. A- 
3), citing Lutz and Hutton (1989), despite the fact that their study presented no empirical data on academic 
achievement of students and presented mixed evidence about the rated performance and retention rates of 
these recruits. 



Two other studies Walsh cites do include student achievement data, but they do not, as she states, compare 
certified with uncertified teachers. Both deal with alternatively certified teachers who receive a substantial 
amount of education coursework while they are undertaking mentored teaching supervised by both university 
supervisors and classroom mentors. 



4. Miller, McKenna, & McKenna (1998) is a matched comparison group study of what the study’s authors call a 
"carefully constructed" university-based alternate route program for middle school teachers. Reflecting the 
characteristics of alternative routes endorsed by the National Commission on Teaching and America’s Future 
(1996), this program offered 1 5 to 25 credit hours of coursework before interns entered classrooms where 
they were intensively supervised and assisted by both university supervisors and school-based mentors while 
they completed additional coursework needed to meet full standard state certification requirements. Forty-one 
of these teachers were compared to a group of 41 traditionally certified teachers matched for years of 
experience, using ratings of their teaching conducted by trained observers. Then student test score data were 
collected for 1 8 of these teachers. Although the sample size is too small to meet Walsh’s criteria (Note 10) for 
studies worth considering (a point she seems to have forgotten here), and data are not provided on student 
pre-test scores, the study appears reasonably well-conducted. 

The traditionally trained teachers in this study felt somewhat more confident in their practice and scored 
slightly higher on the two sub-scales of an observation instrument used by trained observers to rate their 
teaching. However, these differences were not significant, and the authors report, without including the actual 
data analyses, that there were no significant differences in the student achievement of 18 teachers from the 
two groups by the 3rd year of practice after both had completed all of their education coursework. (The 
authors did not control for prior achievement levels of students; however, they stated that the initial 
differences in student achievement across groups were not significant.) 



Because the design of this program was so different from many quick-entry alternative routes. Miller, 
McKenna, and McKenna note that their studies "provide no solace for those who believe that anyone with a 
bachelor's degree can be placed in a classroom and expect to be equally successful as those having completed 
traditional education programs.... The three studies reported here support carefully constructed AC programs 
with extensive mentoring components, post-graduation training, regular in-service classes, and ongoing 
university supervision" (p. 174). This finding does not support Walsh’s contentions throughout her paper that 
only general intelligence and subject matter knowledge make a difference for teacher effectiveness, her 
statement that uncertified teachers do as well as certified teachers, or her claim that there is no evidence 
which supports teacher education and certification. 



5. The other study on alternative certification cited favorably by Walsh (Bradshaw & Hawk, 1 996) was not 
published as a peer-reviewed article or research report — one of Walsh’s criteria for rejecting the results of 
other reports. It is actually not an empirical study but a literature review that, like other reviews Walsh 
criticizes, is based on a mixture of unpublished papers and on studies that, for the most part do not examine 
student achievement. Some of the papers cited do not include empirical evidence at all. Walsh characterizes 
the report’s findings as providing "mixed, inconclusive" evidence. This is certainly true. Studies examining 
measures of knowledge, teacher beliefs and attitudes, teacher ratings, and student views report no differences 
on some measures and differences, typically favoring traditionally prepared teachers, on others, especially 
measures of professional knowledge and performance. 




With respect to student achievement, Bradshaw and Hawk list five papers that discuss outcomes for 



differently trained teachers. The first, an unpublished paper by Barnes, Salmon, and Wale (1989) does not 
present any empirical data or discussion of specific studies, but it includes a statement that two districts in 
Texas reportedly found equivalent outcomes for alternative and traditional program teachers. While it does 
not mention what programs might have been compared, it does include a table listing teacher education 
programs designated as alternatives. This list includes one- and two-year university-based master's programs 
(which are called "alternative" in Texas because they are not undergraduate models) along with district 
alternative programs that generally offer only a few weeks of summer training before teachers are assigned to 
classrooms. Thus, the "alternative" group included programs providing extensive graduate level training of 
the sort that many states would call 'traditional," along with programs that provide little formal preparation. 
Aside from the unanswered question of what analyses some unnamed parties might have been done to 
support assertions about relative effects, the wide range of program models included as "alternative" 
precludes any inferences about the effects of preparation on teacher effectiveness. 



A second study, by Denton & Peters (1988) provides another example of the definitional problems associated 
with the terms "alternative" and "traditional". This paper actually studied two versions of a university's 
college-based teacher education program. The one called "alternative" in their paper was in fact an expansion 
of the regular teacher education program, rather than a reduction in coursework. Graduates of this more 
extensive curriculum had students who had stronger performance in earth and physical sciences, while scores 
in mathematics were stronger for students of the regular teacher education program 



Of the remaining studies, two found that student achievement gains were higher for the students of 
traditionally prepared teachers in language arts (Gomez & Grobe, 1990, in a comparison with alternatively 
certified teachers) and mathematics (Hawk, Coble, & Swanson, 1985, in a comparison with uncertified 
mathematics teachers). The last (Stafford & Barrow, 1994) did not present original research but referenced 
studies reporting differences associated primarily with teaching experience between the performance of 
alternative program teachers, other first-year teachers, and experienced teachers. 



In combination, these studies do not provide any support for the statement that uncertified teachers are as 
effective as certified teachers. In addition to its other inaccuracies, Walsh's review confuses alternative 
certification — a strategy that provides candidates with preparation that is differently packaged from what 
various states deem "traditional" training (usually the difference is that training is post-baccalaureate rather 
than undergraduate and is streamlined into about a year rather than spread across four years of college) — with 
lack of certification — which generally indicates a lack of preparation. Having already missed this critical 
distinction, Walsh does not begin to attempt to sort out the effects of the differences in preparation 
experiences and outcomes associated with different models of teacher education. Thus, she does not note that 
program designs that include a comprehensive and coherent program of coursework and intensive mentoring 
(e.g. Miller, McKenna, & McKenna, 1998) have been found to produce more positive evaluations of 
candidate performance than models that forego most of this coursework and supervised support. 

For example, a comparative study of more than 200 alternative certification candidates in New Hampshire, 
who are certified via three years of on-the-job training in lieu of formal preparation, found they were rated by 
their principals significantly lower than university-prepared teachers on instructional skills and instructional 
planning, and they rated their own preparation significantly lower than did the university candidates 
(Jelmberg, 1995). To understand the outcomes of different approaches, studies of alternatives need to 
acknowledge the differences in program models. 

Finally, Walsh cites two additional studies that include uncertified teachers, but she gets the findings wrong. 
Neither study shows that uncertified teachers do as well as certified teachers. One shows that the reverse is 
true. 

6. In one study (Goldhaber & Brewer, 2000), the authors found that high school students who had a certified 
teacher in mathematics did significantly better^ after controlling for initial achievement and student 
demographic factors, than those who had uncertified teachers. The same trends were true in science, but the 
influences were somewhat smaller. The effects of certification on achievement were larger than — and in 
addition to — the effects of a subject matter degree. In this sample, students of a small number of science 
teachers who held emergency or temporary certification (24 out of the 3,469 teachers in the overall sample) 
did no worse than the students of certified teachers, although they, too, did better than the students of 
uncertified teachers. Another analysis of these data (Darling-Hammond, Berry, & Thoreson, 2001) showed 
that in this sample most of the teachers on temporary / emergency certificates were experienced and most had 
education training comparable to that of the certified teachers. Most appeared to be already licensed teachers 
from out-of-state who were in the transition period to securing a new state license or experienced teachers 
teaching out of their main field. Only a third were new entrants whose characteristics may have suggested a 
content background with little education training. The students of this sub-sample of teachers had lower 
achievement gains in an analysis of co-variance that controlled for pre-test scores, content degrees, and 
experience than those of the more experienced and traditionally trained teachers. 



ERIC 



7. Finally, Walsh cites a recently released study of Teach for America (TFA) by Raymond et al. (2001). This 
study is relevant to Walsh’s discussion of the Resident Teacher Program through which she notes that many 
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TFA recruits enter teaching in Maryland. However, the study did not compare certified to uncertified 
teachers, as Walsh claims. Although they had the data to do so, the authors chose not to examine how TFA 
teachers performed in comparison to trained or certified teachers. The study examined the influences of TFA 
teachers on student achievement scores, using regression methods that controlled for teacher experience and 
school demographics; thus, the comparison was between TFA recruits and other inexperienced teachers in 
high -minority schools in Houston — where most underqualified teachers are placed. Since about 50% of 
Houston’s new hires are uncertified and about 35% were found to lack a bachelors degree in the most recent 
year of the study, TFA recruits were compared to an extraordinarily underprepared set of teachers. In this 
comparison, students of TFA teachers did about as well as those of other inexperienced, largely untrained 
teachers, many of them without bachelors degrees. (Reviewers of this report have noted that the report should 
have compared TFA recruits to other BA holders and to prepared or certified teachers; based on the statistics 
shown, it is not clear that the results of these comparisons would be favorable to TFA.) (Note 1 1) Another 
study that compared TFA teachers to certified teachers found significantly higher scores for the students of 
certified teachers (Laczko-Kerr and Berliner, 2002). The Raymond et al. report also indicated that minority 
students in Houston, who are disproportionately taught by these underprepared teachers, lose ground 
academically each year. In addition, only about 50% of African American and Latino 9th graders in Houston 
graduate from high school four years later (Haney, 2000; NCES, 2000). It would be hard to argue that the 
assignment of so many underprepared teachers to these students has nothing to do with their lack of success. 

The TFA study found that students of experienced teachers performed significantly better than students of 
inexperienced teachers, including TFA recruits. Along with the report’s finding that, over a three year period, 
between 60% and 100% of TFA candidates had left after their second year of teaching, this finding raises 
additional questions about Teach for America’s contribution to the education of Houston students, since they 
do not stay long enough to gain the experience that could support student achievement. Earlier data from the 
Maryland Department of Education showed that TFA recruits in Baltimore had similar attrition rates, with 62 
% gone by the third year of teaching (Darling-Hammond, 2000b). 

These high attrition rates resemble those found in some other studies of short-term alternative routes 
(Darling-Hammond, 2000c) and suggest another important outcome of teacher preparation policies. Both the 
Houston study and Walsh’s own review indicate that experienced teachers are more effective than 
inexperienced teachers (Walsh, pp. 5-6), yet many short-term alternative program recruits leave quickly. 

Other research indicates that those who complete 5-year teacher education programs enter and stay in 
teaching at much higher rates than 4-year teacher education graduates, who stay in teaching at higher rates 
than teachers hired through alternatives offering only short-term summer training before full-time teaching 
(Andrew & Schwab, 1995; Darling-Hammond, 2000b). One reason for this might be the fact that 5-year 
program graduates typically have both a disciplinary major and a full-year of student teaching tightly 
integrated with education coursework. 

Student teaching appears to make a strong difference in teacher retention. In a longitudinal study of recent 
college graduates who entered teaching in 1993, a recent NCES report notes that recruits without student 
teaching — most common among untrained recruits or those who enter through shorter-term alternative 
routes — leave teaching at rates nearly twice as high as those who have had this kind of clinical training 
(Henke, Chen, & Geis, 2000). The authors noted: 

In comparison with new teachers who had less training in pedagogy, those with more 
training were less likely to have left teaching without returning by 1997. Fifteen percent of 
those who had student taught had left the profession and not returned by 1997, compared 
with 29 percent of those who had not student taught. Where as 14 percent of certified 
teachers had left by 1997, 49 percent of those without certification had not done so (p. 49). 

Findings about the high attrition rates of those hired without full preparation for teaching raise questions 
about the cost-effectiveness of a recruitment strategy that relies on teachers with little preparation who are 
likely to leave the profession before they can learn to become effective with children. Meanwhile, the 
children they have taught — almost always the most disadvantaged students in the most disadvantaged 
schools — have not had the benefit of a teacher with either professional knowledge or experience — ^two 
sources of greater teaching skill. 

A recent study in Texas showed that teacher attrition costs school systems at least $8,000 for each recruit who 
leaves in the first few years of teaching (Texas Center for Educational Research, 2000). It estimated that the 
high attrition of beginning teachers in Texas, a growing number of whom enter with little or no preparation 
and receive few supports in learning to teach, costs the state more than $200 million per year (p. 16). This and 
other studies of teacher attrition suggest that policymakers should consider both teaching effects and retention 
patterns when they think about how to recruit and prepare teachers. 

Walsh chooses to ignore other studies showing that certified teachers do better than uncertified teachers. 

8. One of these by Hawk, Coble, & Swanson (1985), entitled ’’Certification: It Does Matter,” found — in 
contradiction to Walsh’s statement cited above — that teachers’ certification in mathematics has a large and 




statistically significant effect on student achievement gains in both general mathematics and, to an even 
greater extent, in algebra. It compared pre- and post-test scores of students whose teachers who were certified 
in mathematics as compared to those of teachers with similar levels of experience who were uncertified in 
mathematics. This study is dismissed in one part of Walsh's review as too small (p. 34), so that its findings 
can be discounted with respect to certification. However, the size of the study does not appear to matter to 
Walsh when she chooses to cite it as a basis for arguing that only subject matter makes a difference to 
teaching effectiveness (p. 65). This double standard about the use of research permeates the report. A study is 
declared inadequate when it finds any contribution of teacher education or certification to any measure of 
teacher effectiveness but a study of comparable size or methodology — often the same study — is embraced 
elsewhere and used to support a different argument. 

While the study does have a small sample size (it examined 36 teachers, paired by school, course, and ability 
level of students being taught and the 826 students they taught), it is a reasonably well-controlled matched 
comparison design. The study does support the idea that subject matter knowledge matters to teaching. 
However, Walsh misrepresents the study as suggesting that only subject matter knowledge matters. The study 
did not directly examine the isolated effects of subject matter knowledge but the combined effects of subject 
matter knowledge and educational knowledge — including methods courses in the teaching of the content 
area — that are part of the certification requirements for an in-field credential. Authors Hawk, Coble, and 
Swanson concluded: 



The results of this study lend support to maintaining certification requirements as a 
mechanism to assure the public of qualified classroom teachers... " (p. 15). (Note 12) 



As this and other studies reviewed here suggest, content knowledge in combination with content pedagogical 
knowledge — that is, knowledge about how to teach the content, which, together with student teaching, 
constitute the major components of certification — appear to make contributions to student learning that 
exceed the contributions of either component individually. An important policy point from this and other 
studies of certification is the fact that teachers would not have been guided or encouraged to acquire the 
content knowledge and content pedagogical knowledge represented by in-field certification unless there were 
certification requirements. While Walsh and the Fordham Foundation manifesto she endorses would turn all 
hiring decisions over to principals, it was principals in these schools — and in many others across the 
country — who hired and assigned out-of-field teachers to teach mathematics as well as other subjects 
(Ingersoll, 1998). In a policy world that eliminates teacher certification, there would be no barrier to that 
practice occurring on an even more widespread basis. 



9. Another, much larger study resulted in similar findings about teacher certification in California. Fetler (1999) 
examined the relationship between school scores on the state's mathematics test and teachers' average 
experience levels and certification status in 795 high schools, after controlling for student poverty rates and 
test participation rates. It found that the percent of teachers on emergency credentials exerted a strong and 
highly significant negative influence on student achievement. The author concluded that, "After factoring out 
the effects of poverty, teacher experience and preparation are significantly related to achievement" (p. 13). 

This study is cited but never discussed in Walsh's revised report. In her original appendix, Walsh applauded 
the study's methods but then sought to dismiss its findings with two inaccurate assertions. First, she 
suggested, incorrectly, that the study's results pertained to subject matter knowledge alone, not to the 
combination of subject matter and teaching knowledge represented by certification. She misread both the 
study and the requirements of California's credentialing system to make this claim, appearing to believe that 
individuals who have passed only the subject matter requirement of a content test are granted full credentials 
in California (they are not), that individuals who are certified through internship programs (California's 
alternative route) do not have to complete pedagogical requirements (this is false), and that individuals are 
hired on emergency permits solely if they lack content knowledge (this is also false). (Note 1 3) Walsh also 
suggested, incorrectly, that the study "may have some basic methodology problems, by reaching conclusions 
using aggregated state-wide data." However, all of the study's data are aggregated to the school level, not the 
state level. (See the author's confirmation of this statement, below.) In the original appendix, (Note 14) Walsh 
stated; 



Tlie article would be only be of interest if someone tried to assert that a teacher who knows 
no math could be a good math teacher. Any attempt to use this study as evidence against the 
practice of hiring alternatively trained teachers, as appears to be Darling-Hammond's implies 
(sic) and as Wilson et al. interpret it, loses all of its impact after reading Fetler.... In fact the 
author.... is primarily advocating ensuring that math teachers take more subject matter 
coursework, and is clearly disinterested in any effect that may be had from coursework in 
"professional knowledge." 

The author, Mark Fetler, took strong issue with this interpretation of his findings. When I shared Walsh's 

statement with Fetler, he wrote in reply: 

I am surprised that Kate Walsh makes those statements. I had a brief telephone conversation 




with her, but she was not forthcoming about her intent. Meeting the subject matter 
requirement involves both knowing the topic, e.g.. Algebra, and the specific procedures 
needed to teach it in the classroom. Someone who knows how to solve quadratic equations, 
but does not know how to convey that information to children in a classroom, is a poor 
teacher. Both math subject knowledge and math pedagogy are essential. I believe that my 
study is consistent with these statements.... I would be surprised to hear of any research that 
demonstrated successful teaching that lacked either of those elements. My study supports the 
importance of appropriate credentials. Supposing that you could find people who know math 
to teach, if they lack the ability to communicate effectively with children, they will not 
succeed in the classroom and will create dissatisfied students, parents, colleagues, 
administrators, and board members. It will be a mess. Higher standards, not lower, are the 
solution. 

Fetler also noted that, “the unit of analysis in my paper is the school. It is not based on statewide aggregated 
data." 

Two other recent school-level studies in California have found significant negative relationships between 
average student scores on the state examinations and the percentage of teachers on emergency permits, after 
controlling for student socioeconomic status and other school characteristics (Betts, Rueben, & Dannenberg, 
2000; Goe, forthcoming). Like Fetler's study, these studies also found smaller positive relationships between 
student scores and teacher experience levels, with negative effects on student achievement associated with the 
proportion of beginning teachers. 

California’s experience is a good example of what happens when pressures and supports for hiring 
credentialed teachers are relaxed. After nearly a decade of inadequate and unequal salaries, easy access to 
emergency permits and waivers, and few incentives for the training and equitable distribution of qualified 
teachers for high-need fields and locations, California, now one of the lowest-achieving states in the nation, 
found itself with more than 40,000 teachers teaching on emergency permits or waivers by 1999-2000. The 
vast majority of these teachers were teaching in a small number of urban school systems in schools with the 
highest proportions of low-income students and students of color. High-minority schools were nearly seven 
times as likely to have uncredentialed teachers as low-minority schools. Low-achieving schools were nearly 
five times as likely to have uncredentialed teachers as high-achieving schools (Note 15) (Shields et al., 2000, 
pp. 41-43). 

These results mirror those already noted in Baltimore, Houston, and other cities. The pattern appears across 
the country. For example, a recent series in the Chicago "Sun Times" (Note 16) documented that "children in 
the state's lowest-scoring, highest-minority and highest-poverty schools were roughly five times more likely 
to have teachers who had flunked at least one certification test" and were least likely to have teachers who 
were "correctly certified." The burden should be on those who argue against efforts to ensure minimally 
qualified teachers for all students to prove that the confluence of race, poverty, and low achievement with the 
presence of untrained and uncertified teachers does not further disadvantage our nation’s most vulnerable 
students. 

Evidence about Preservice Teacher Education 

For the proposition that "there is little evidence that the content and skills taught in preservice education coursework 
is (sic) either retained or effective" (p. 7), Walsh cites two articles (Mumane, 1983; Veenman, 1984) from among the 
many dozens of studies of teacher education that could have been retrieved from the peer-reviewed literature, had she 
done a search. Both of these are very old pieces, published long before recent reforms in teacher education. Neither of 
them makes any statement in support of Walsh's claim. 

1 . Veenman ( 1 984) describes the most frequently cited problems by novice teachers. These included concerns 
about topics ranging from classroom management to teaching loads and class sizes. Nowhere in the article 
does he suggest that what teachers learned in preservice education was not retained or effective. In fact, he 
notes that researchers should look more to the conditions of schooling than to teacher education for 
explanations for many of the problems beginning teachers cite. Veenman notes that the outcomes of teacher 
education may vary by characteristics of programs, citing studies finding that those who had had more intense 
student teaching, more competency-oriented teacher education coursework, or who were more satisfied with 
their teacher education experiences reported fewer problems in the classroom. 

2. Mumane's (1983) article is not an empirical study but a brief commentary on the work of another author who 
proposed the development of doctoral degrees for teacher leaders. While he questions the value of doctoral 
education for developing pedagogical skills (as would I), Mumane is careful to point out that there are forms 
of teacher education that may be helpful, and that lack of evidence in large data sets about the effects of 
preservice education may be related to the lack of data collected on the topic at that time, nearly 20 years ago. 
(See additional discussion of this point under "Evidence about Verbal Ability" below.) 




3. Walsh ignores the findings of other studies on this topic, including some she has cited for other propositions. 
She criticizes Evertson, Hawley, and Zlotnik (1985) for their interpretion of the findings of Edward Begle 
(1979), "a respected mathematician" regarding his findings about teachers' subject matter preparation (p. 34). 
In one of the few early data sets providing evidence about teacher preparation — a mammoth study of 1 1 2,000 
students conducted through the National Longitudinal Study of Mathematical Abilities — Begle (reported in 
Begle & Geeslin, 1972 and, with additional data, in Begle, 1979) found that measures of teacher subject 
matter knowledge did not exert strong influences on student achievement. He also found that coursework in 
mathematics methods had a stronger effect on student achievement than higher-level coursework in the 
subject matter (discussed in Begle, 1979). On the lack of influence of subject matter knowledge in his earlier 
study (Begle & Geeslin, 1972) Begle noted, and Walsh reports, that the teachers in the study may have had 
stronger content knowledge than the norm, since they had all been accepted to a National Science Foundation 
Summer Institute. This is an appropriate point. 



However, Walsh chooses to ignore Begle's findings about the value of education coursework. She does not 
explain why. Walsh cites Begle's work at several points in her text, and refers readers to her appendix for a 
review of his work that is no longer there. In her separately-published appendix, Walsh admits of Begle 
(1979) that, "this is a scholarly work, employing defensible analyses at the time it was written for examining 
the data." She then nonetheless sought to dismiss it with a vague statement about possible aggregation bias 
(although achievement data were aggregated only to the classroom level), "too many variables" in the data 
set, and "much greater variance in the number of subject matter courses teachers took than the number of 
methodology courses they took." This last complaint is particularly odd. The implications of greater 
variability in subject matter courses contradicts the point she makes above about the possibly high levels of 
subject matter knowledge among sample members (in re: Begle & Geeslin, 1972). In fact, wider variability 
would generally make it easier to find effects, if they are there to be found, rather than harder. In another 
instance (regarding Byrne, 1983), Walsh notes, correctly, that the limited variability in subject matter 
coursework levels may have made effects more difficult to find. Walsh seems confused about the research 
findings and their implications but clear about her goal of discrediting any results that support the value of 
teachers learning about how to teach their content to others. 



4. Monk (1994) offers similar findings on this question from a more recent data set that incorporates more fine- 
grained variables about teacher education. Using data on 2,829 students from the Longitudinal Study of 
American Youth, Monk (1994) found that teachers' content preparation, as measured by coursework in the 
subject field, is positively related to student achievement in mathematics and science, but he notes that the 
relationship is curvilinear, with diminishing returns to student achievement of teachers' subject matter courses 
above a threshold level (e.g., five courses in mathematics). In addition, teacher education coursework (e.g. 
methods courses in the content area) had a positive effect on student learning in mathematics, exhibiting 
"more powerful effects than additional preparation in the content area" (p. 142). Monk concluded that "a good 
grasp of one's subject area is a necessary but not a sufficient condition for effective teaching" (p. 142). 



Monk told me that when Walsh first shared her brief appendix review of his work with him, he was surprised 
that she had used his work to emphasize the importance of subject matter knowledge without acknowledging 
his findings on the value of education courses. He noted in an email to me that he had communicated to 
Walsh that: 



My study of relationships between teacher course taking experiences and subsequent student 
gains in performance showed that the number of both content courses and content-specific 
pedagogy courses in a teacher's background is positively related to pupil test score gains in 
the relevant content area. It is misleading to report the positive results for the content courses 
and to not acknowledge the positive results for the pedagogy courses. 

After Monk communicated with Walsh, she did acknowledge in her appendix that Monk's study provides 
support for the contention that education coursework has a positive effect on teaching performance; however, 
she did not incorporate this admission in her claims that "not one" of the studies ever cited on this topic 
provides such support. 

5. In addition to newer databases that allow some large-scale examinations of the influences of teacher 

education variables on student achievement, recent studies have begun to look at the outcomes of different 
teacher education program designs. For example, studies of 5-year teacher education programs — programs 
that include a bachelor's degree in the discipline plus an additional year of education study and extended 
student teaching — have found graduates to be more confident and better rated than graduates of 4-year 
programs in the same institutions and as effective as more senior teachers, as well as more likely to enter and 
remain in teaching (Andrew & Schwab, 1995; Denton & Peters, 1988). Walsh does not review or cite any of 
these studies, even those that were available for her information from previous research she claims to have 
scrutinized. 

The Influence of Verbal Ability on Teacher Effectiveness 

There is little disagreement about the fact that verbal ability and subject matter knowledge influence teacher 



effectiveness, although Walsh tries to set up a straw man by suggesting, inaccurately, that some researchers, including 
myself, have argued otherwise. (See the section on ’’Misrepresentations of Research" below.) There are two areas of 
real disagreement, however. One is whether verbal ability alone is the only or best measure of teacher effectiveness. 
The other is how to evaluate the size of relative contributions of various kinds of knowledge to teacher effectiveness. 



As examples cited earlier illustrate, the literature on teacher characteristics and their effects on teacher performance 
has been a captive of the measures most likely to be available in large data sets at any moment in time. While there 
are many studies evaluating the influences of teachers' standardized test scores, especially measures of verbal or 
general academic ability, because these variables have been readily available in large-scale data sets since the 1960s, 
data on teachers' course-taking backgrounds or teacher education experiences have been included in large data sets 
only since the early 1990s. Thus, there are more studies finding influences of variables that have most often been 
measured. 



Finally, most of the studies that have included measures of verbal ability or content knowledge have not included 
measures of teacher education or certification. In a recent review, Wayne and Youngs (in press) found five studies 
that observed relationships between measures of teachers’ verbal or general academic ability and student achievement 
and that met the standard of having controlled for students' socioeconomic status and prior achievement. Four of these 
studies employed data sets from the 1960s and 1970s and none of the five included measures of teacher education or 
certification. Looking across studies in these different eras, in many cases, the relative effect sizes of verbal ability 
measures are no larger than those of teacher education and certification measures in the studies that use these instead, 

1. Walsh uses an article by Mumane (1983) written nearly 20 years ago to argue for the primacy of verbal 
ability as a correlate of teacher effectiveness. She states, illogically, that, "to concede this relationship would 
mean acknowledging that formal teacher preparation is not as critical to student achievement as some would 
advocate" (p. 41). However, Mumane pointed out in his article that evidence about the influence of verbal 
ability was partly a function of the fact that teachers' standardized test scores were one of the few variables 
about teachers available in large-scale databases at that time, which did not include good measures of teacher 
education. In discussing the results on verbal ability, he diverges from Walsh's interpretation, stating: 

Clearly one should not interpret these results as indicating that intellectual ability should be 
the sole criterion used in recruiting teachers or that formal teacher training cannot make a 
difference. In fact, the lack of evidence supporting formal preservice training as a source of 
competence may be to some extent a result of limitations in the available data. For example, 
all databases suitable for examining the correlates of teaching effectiveness as measured by 
student achievement gains pertain to a single school district. Since there is less variation in 
training among teachers within a district than among teachers in the country at large, these 
databases do not permit the most powerful possible tests of the efficacy of alternative teacher 
training programs (p, 565), 

2. Walsh tries to use another article by Greenwald, Hedges, and Laine (1996) as evidence that verbal ability is 
the only critical variable influencing teacher effectiveness, and misrepresents a communication she had with 
Larry Hedges, one of the study's authors, regarding the appropriate interpretation of his findings. 
Characterizing Greenwald, Hedges, and Laine's article as "a sound review of 60 studies,’' she then criticizes a 
direct reference to its findings in a report by the National Commission on Teaching and America's Future 
(Walsh, p. 17), Her criticism first alludes, incorrectly, to a chart in the Commission’s report (which in fact 
referred to another study, (Note 17)) then she criticizes the interpretation of the chart. The correct chart in the 
Commission’s report (Figure 5, entitled "Effects of Educational Investments" in Darling-Hammond, 1997, p. 
9) was reproduced directly from Greenwald, Hedges, and Laine’s table 7, column 1 (p, 379) with the same 
variable labels and statistics as presented in the original source. It describes the size of increase in student 
achievement for every $500 spent on several different kinds of investments. Here is a reproduction of the 
table from Greenwald et al.’s study: 



Table 7 

The effect of $500^ per student on achievement^ 





Sample 


Input Variable 


Full Analysis 


Publication bias robustness 


Per pupil expenditure 


0,15 


0,15 


Teacher education 


0.22 


0,20 


Teacher experience 


0,18 


0,17 


Teacher salary 


0.16 


0,08 



180 



Teacher/pupil ratio 



0.04 



0.04 



®1993-94 dollars 

•’All achievement outcomes are in standard deviation units. 

In explaining the table, study authors noted that 

The magnitudes (of the effects) for teacher education and teacher experience are higher than, 
but of the same magnitude, as PPE (per pupil expenditures). That is, one would expect 
comparable and substantial increases in achievement if resources were targeted to selecting 
(or retaining) more educated or more experienced teachers, (p. 380) 

The Commission used this finding, as Greenwald, Hedges, and Laine had done, as an indicator that 
investments in teacher education showed stronger influences on pupil achievement gains than investments in 
other resources, like reduced teacher/pupil ratios. We noted in discussing their overall study that the authors 
had found evidence of the influences of teacher ability and experience, along with teacher education. 
However, Walsh criticizes the Commission’s two-sentence characterization of the research (which she calls a 
discussion "in considerable detail") for failing to note that Greenwald, Hedges, and Laine found more studies 
supporting the influences of teacher verbal ability on achievement than what they labeled "teacher 
education" (measured in their study as masters degrees because this was the most widely used measure in 
large data sets.) She suggests that Hedges disagrees with the Commission's characterization, a view that 
Hedges clarified was inaccurate when I spoke to him. He indicated that Walsh had not revealed her 
interpretation of his findings when she contacted him, and wrote the following to explain his own view of the 
proper interpretation of his findings: 

It is true that the relationship between teacher verbal ability and student achievement is 
relatively large and consistent across the few studies that have examined it. However this 
does not imply that investing in teacher ability (among possibly poorly qualified teachers) is 
a cost effective way to enhance student achievement. There are two reasons. First, teacher 
ability (among qualified teachers) may be more expensive than other resources that could be 
purchased to improve achievement. That is, there could be a strong relationship but high 
cost. Second, and more important, the relations found in the studies Greenwald, Hedges, and 
Laine (1996) reviewed were studies of practicing teachers. There is no reason to expect that 
the same relation holds among those who are not part of the teaching workforce. 

The point here, similar to that made by Mumane (above), is not that verbal ability is not important, but that 
the evidence does not prove it is the only important contributor or the most efficient way to achieve teacher 
effectiveness. In fact, most current certification systems combine tests of basic skills and general academic 
ability, subject matter, and teaching knowledge with evidence of successful supervised clinical experience 
and coursework focused on teaching knowledge and skills to help candidates assemble many sources of 
expertise in a more coherent way than would otherwise be the case. 

In pursuit of her argument that only verbal ability makes a difference, Walsh seeks to discount other studies 
that have found strong influences of teacher certification test scores on teacher effectiveness as being relevant 
only to the measurement of verbal ability and irrelevant to the broader question of teacher certification. These 
studies are also misrepresented. 

3. In her discussion of Schalock (1979) in the appendix (B13), Walsh seeks to dismiss his review’s findings 
about the limited evidence regarding the relationships between teachers’ measured intelligence and other 
indicators of effectiveness because the review is "old, old!!" and because, she argues, "More recent research 
such as Summers and Wolfe, 1977; Ferguson, 1991; Ferguson & Womack, 1996 (sic)\ Mumane, 1983; 
Hanushek, 1971; Strauss and Sawyer, 1986 suggest that intelligence (measured by SAT, verbal ability tests 
and college selectivity) are indeed substantially important." 

Aside from the facts that two of these "more recent" studies pre-date the review she dismisses as "old, old!" 
and one (Mumane, 1983) is not a study at all, Walsh here cites two studies that she dismisses elsewhere for 
"aggregation bias" (Ferguson, 1991 and Strauss & Sawyer, 1986, see Walsh, p. 27) and another (Ferguson & 
Womack, 1993) that she dismisses without stating a reason (see discussion of Wilson et al., in Appendix B). 
(Note 1 8) Walsh's readers are referred to Appendix B for reviews of these issues, but the studies are not 
included there. 

4. Walsh cites Ferguson ( 1 99 1 ) for a number of her propositions, including the fact that teacher quality matters 
(p. 5), that teacher race does not matter (p. 6), and that verbal ability matters (p. 6). Later, she claims — when 
she wants to dismiss the study for its findings about teacher education and certification — that the study 
suffers from aggregation bias, a concern I address in the next section on methodological issues. Ferguson’s 
analysis of nearly 900 Texas school districts controlled for student background and district characteristics; he 





found that combined measures of teachers' expertise — scores on a state teacher licensing examination, 
master's degrees, and experience — accounted for more of the inter-district variation in students' reading and 
mathematics achievement (and achievement gains) in grades 1 through 1 1 than student socioeconomic status. 
An additional, smaller contribution to student achievement was made by lower pupil-teacher ratios and 
smaller schools in the elementary grades. The effects were so strong, and the variations in teacher expertise so 
great, that after controlling for socioeconomic status, the large disparities in achievement between black and 
white students were almost entirely accounted for by differences in the qualifications of their teachers. 



As I noted in an earlier review of this study (Darling-Hammond, 2000c), of the teacher qualifications 
variables, the strongest relationship was found for scores on the TECAT, a state licensing examination 
described by the test developer as a test that measures basic skills and professional knowledge. The Texas 
Education Agency's published outline of the test content shows that it seeks to measure verbal ability, logical 
thinking, research skills, and a set of items on professional knowledge. Walsh takes issue with this description 
of the test and argues that the study does not support the value of teacher certification because the test should 
be considered primarily a basic literacy test. In Walsh's view, this makes it irrelevant to the question of 
teacher certification — even though it is required for teachers to maintain their certification. She also argues 
that the relatively smaller influence of master's degrees in Ferguson's study (which accounted for about 5% of 
the explained variance) means that teacher education is unimportant, and she criticizes the fact that I discuss 
the three variables associated with teacher quality (TECAT scores, experience, and masters degrees) in 
combination, although this is also the way in which Ferguson discusses them at several points in his analysis. 



Walsh's arguments are illogical in several ways. First, while it is true the TECAT measures basic skills, it also 
measures other academic abilities and professional knowledge, as confirmed by the test maker's 
documentation and administering agency's descriptions. There is no basis for making judgments contrary to 
the claims of the developers. In addition, the test would not exist at all if there were not a state certification 
system requiring it. Like all of the other variables one can evaluate in studies of this kind, the test scores are a 
rough proxy for many aspects of teacher capacity that may matter for their performance. In a regression 
equation of this sort where one variable stands in for others for which data are not available, it undoubtedly 
captures the effects of other unmeasured factors. Even if it were true that the test was a weak measure of 
professional knowledge, this would not mean that professional knowledge is unimportant or that verbal 
ability is the only important variable for predicting teaching ability. Only a better measure of professional 
knowledge (coursework or a more in-depth test of teaching knowledge) would allow a test of this question. 
Finally, as Hedges notes above, since the Ferguson study was based on practicing teachers, its findings do not 
shed light on the relative effectiveness of non-teachers who might score differently on the tests. 



Masters degrees and experience are other very partial measures of teacher knowledge and skill that show a 
modest effect in this study and a larger effect in Ferguson and Ladd's (1996) similar study in Alabama that 
included a weaker test measure of pre-college general skills (the ACT), which is not designed to capture 
knowledge relevant to teaching. However, masters degrees are also a very crude proxy for teacher education, 
given the wide variability in the content of masters degrees pursued by teachers, many of which have been 
pointed at jobs outside of teaching, such as administration, counseling, measurement and evaluation. In fact, 
aside from MAT preparation programs in a small number of institutions and specialist programs for reading 
and special education, there were few masters degree programs for the study of teaching until the recent 
advent of 5-year teacher education programs and masters degrees developed around the National Board for 
Professional Teaching Standards that focus on content pedagogy. Thus, there is reason to expect that some 
masters degree studies would affect teaching ability, but not much reason to expect the effect of masters 
degrees as an undifferentiated variable to be uniform or large in the aggregate, a point I have made in earlier 
commentary (Darling-Hammond, 2000a). Goldhaber and Brewer (1998, 2000) have made the same point and 
have completed research that documents the greater influence of both bachelors and masters degrees in the 
content area taught (e.g. mathematics or mathematics education) as compared to undifferentiated degrees. 



It makes more sense to consider these variables together as proxies for expertise than to treat them as 
mythically precise measures of totally unrelated constructs. As I have argued elsewhere, research on teaching 
suggests a view of expertise that includes general knowledge and ability, verbal ability, and subject matter 
knowledge as foundations; abilities to plan, organize, and implement complex tasks as additional factors; 
knowledge of teaching, learning, and children as critical for translating ideas into useful learning experiences; 
and experience as a basis for aggregating and applying knowledge in nonroutine situations (Darling- 
Hammond, 2000a). David Berliner's studies of expertise in teaching, for example, include experience along 
with several other traits as a critical aspect of expertise (see e.g. Berliner, 1986). All of these factors combine 
to make teachers effective; furthermore, one cannot fully partial out the effects of one factor as opposed to 
another as many are highly correlated. 

5. Walsh also cites Strauss and Sawyer (1986) for her proposition that verbal ability matters (p. 6), but fails to 
report the study's actual findings and seems unconcerned that it might suffer from "aggregation bias." In a 
study of 145 school districts in North Carolina, these researchers found that teachers' average scores on the 
National Teacher Examinations (NTE) had a strong influence on average school district test performance. 
Although the authors did not specify which portion(s) of the NTE were used as measures, the Weighted 
Common Examinations Test (WCET) was required in North Carolina at that time The WCET included 
separate subtests measuring general knowledge and professional knowledge about teaching. Walsh apparently 
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wants to count this as a test of verbal ability, but does not acknowledge the Professional Knowledge 
Examination portion of the test. 



The authors found that, taking into account per-capita income, student race, district capital assets, student 
plans to attend college, and pupil/teacher ratios, teachers’ certification test scores had a strikingly large effect 
on students’ failure rates on the state competency examinations: a 1% increase in teacher quality (as measured 
by NTE scores) was associated with a 3 to 5% decline in the percentage of students failing the exam. The 
authors’ conclusion is similar to Ferguson’s (1991): 

Of the inputs which are potentially policy-controllable (teacher quality, teacher numbers via 
the pupil-teacher ratio and capital stock), our analysis indicates quite clearly that improving 
the quality of teachers in the classroom will do more for students who are most educationally 
at risk, those prone to fail, than reducing the class size or improving the capital stock by any 
reasonable margin which would be available to policy makers (p. 47). 

The same illogic holds in regards to the dismissal of this study as the previous one. 

In addition to questions about the content of tests used in various studies, the measures that appear in large 
data sets are always relatively crude proxies for the constructs under study, so it is impossible to know with 
great precision exactly what trait is being represented when a variable shows an effect. For example, scores 
on tests of academic ability like the SAT have generally been strongly correlated with scores on ETS subject 
matter and professional knowledge tests (Gitomer, Latham, and Zimek, 1999); in eras when higher degrees 
were less common (e.g. pre-1980), verbal ability scores were also strongly correlated with masters degrees. 
Where certification tests are in place, test scores correlate with certification status. And both certification 
status and masters degrees typically correlate with teacher experience, since most states require teachers to 
obtain certification in order to remain in the workforce and most teachers have traditionally secured masters 
degrees by taking courses over time while teaching. (This is changing to some extent where beginning 
teachers are being trained in post-baccalaureate or 5 -year programs and sometimes enter the workforce with a 
masters degree). 

These interrelationships do not invalidate studies that have used one or more of these variables, but they are 
one reason why it is difficult to say with certainty which of these measures — or other unmeasured variables 
that are related to them — are associated with measured effects. The correlational studies that Walsh relies on 
almost exclusively do not establish causation; they point to possible relationships for further, more fine- 
grained exploration. However, Walsh often dismisses other large studies and the more fine-grained studies 
from consideration, at least when the findings do not suit her predilections. 

6. Walsh also cites Ferguson & Womack (1993) for her proposition that verbal ability matters most, although 
the reason for this is unclear. This study of more than 250 candidates from a single teacher education program 
examined the influences on 13 dimensions of teaching performance of education and subject matter 
coursework, NTE subject matter test scores, and GPA in the student's major. The ratings of performance were 
based on detailed descriptors of teaching on 107 items evaluated by subject matter specialists and education 
supervisors. The authors found that the amount of education coursework completed by teachers explained 
more than four times the variance in teacher performance than did measures of content knowledge (NTE 
specialty scores and GPA in the major). It is possible that Walsh cites this study as support for verbal ability 
influences because she has confused the NTE specialty tests of subject matter knowledge with other 
components of the NTE battery measuring general academic ability. In any event, the strength of the 
relationship was very small. Given her willingness to cite the study for a very weak finding about verbal 
ability, it is interesting that she does not cite it for its much stronger finding that education coursework 
mattered for teaching performance. 

In her separately-published appendix, Walsh seeks to dismiss the Ferguson & Womack study because it is 
limited to a single institution (Note 19) and uses ’’supervisor’s evaluations” as the measure of performance. As 
noted earlier, she is willing to use studies based on such measures for her own claims, despite her assertions 
that they should not be included. More important, in this study the ratings are not the global ratings from 
school principals that have often been found to be relatively low in reliability. They are lower-inference 
ratings based on a detailed protocol used by subject matter specialists and university supervisors, which are 
typically more reliable. In addition, the limitations on generalizability created by the use of a single institution 
are not fatal to consideration of the findings. They require that the study be considered in the context of other 
studies on similar questions using different samples. Such studies have been conducted. 

7. In a similar study which compared relative influences of different kinds of knowledge on 12 dimensions of 
teacher performance for more than 270 teachers, Guyton and Farokhi (1987) found consistent strong, positive 
relationships between teacher education coursework performance and teacher performance in the classroom 
as measured through a standardized observation instrument (the Georgia Teacher Performance Assessment 
Instrument), while relationships between classroom performance and subject matter test scores were positive 
but insignificant and relationships between classroom performance and basic academic skill scores were 
almost nonexistent. (The two measures of basic academic skills were the Georgia Regents’ test, a required 




examination for public university students, for which the researchers used reading and essay scores, and the 
states’ Teacher Competency Test.) 



The researchers noted that extensive reliability studies had been conducted to support the reliability of the 
TPAl performance measure, which was used statewide as an assessment for certification. Walsh eliminates 
this study from consideration because it is a single institution study and refers the reader to Appendix B for 
her review (p. 25). In her appendix, Walsh criticizes the study for its reliance on supervisors' ratings, again 
failing to distinguish the research on principals' general teacher evaluation ratings from the research on the 
reliability of the TPAl as an observational instrument. She also apparently failed to read the study carefully, 
questioning why the numbers of teachers differ for various comparisons, not having noted the authors' 
explanation that all correlations depended upon the number of teachers for whom data on both variables were 
available (p. B1 1). 

Whereas Walsh tries to paint an unambiguous picture about the value of such measures as verbal ability 
(suggesting, for example, that these scores be reported statewide as a primary measure of accountability) and 
the lack of value of teacher education, the real picture is decidedly more complex. Her evidence for her 
claims confuses measures of verbal ability with measures of professional knowledge and subject matter 
knowledge, and often includes studies that actually show influences of these other kinds of knowledge that 
are at least as strong as measures of verbal ability. The world is just not as simple as Walsh would like to 
make it appear. Even strong advocates of the notion that academic ability matters are not willing to make the 
kinds of over-assertions Walsh urges. For example, Hanushek (1992), whom Walsh cites repeatedly for her 
defense of verbal ability as a key measure concludes: 

The closest thing to a consistent finding among the studies is that "smarter" teachers who 
perform well on verbal ability tests do better in the classroom. Even for that the evidence is 
not very strong (p. 1 16). 

While it would be ridiculous to argue that verbal ability and subject matter knowledge do not matter for 
teaching, it is equally ridiculous to argue that knowledge of teaching and learning and the opportunity to learn 
to teach under the close supervision of a master teacher through student teaching and other guided 
experiences do not matter at all. The literature just does not support this reading or the policy implications 
that Walsh would draw. 

The Academic Ability of Teachers who Lack Certification 

Another argument made by those who would eliminate certification is that an unconstrained market would allow the 
recruitment of individuals with higher verbal or general academic ability who do not now enter teaching. While it is 
probable that some individuals would choose to teach if they did not have to prepare, it is not clear that most of these 
entrants would be more academically able, that they would be better teachers, or that they would stay long in 
teaching. It is also unlikely that given current wages, individuals who are now preparing for much higher-paying 
careers in medicine, the law, engineering, and other professions that require much more onerous preparation and 
licensing processes would choose teaching as a career simply because they did not have to be certified. 

Labor market contexts are relevant to this question. The qualifications of individuals preparing for teaching improved 
noticeably between the early 1980s and the early 1990s in terms of both academic attainment and ability measures, in 
part because of the changes in admissions requirements to teacher education adopted by states and universities but 
also likely because of the substantial increases in real wages for teachers that occurred during the 1980s. Whereas 
prospective teachers were disproportionately drawn from the bottom quartile of college students in the early 1980s 
(Lanier & Little, 1986), both grades and test scores improved for teacher candidates by the 1990s. 

The Recent College Graduates Survey, which tracks college graduates into the labor market, found that the grade 
point averages of newly qualified teachers in 1990 were higher than those of the average college graduate, with 51% 
earning a GPA of 3.25 or better as compared to 40% of all graduates (Grey et al., 1 993). However, average GPAs 
were significantly lower for the 15% of college graduates entering teaching who were neither certified nor eligible for 
certification. Most of the uncertified entrants (57%) had grade point averages below 3.25, and 20% had GPAs below 
2.25. Attrition was also high for the untrained candidates. By the time of the survey (one year later), only one-third of 
the uncertified entrants were still engaged in teaching as their primary jobs (Grey et al., 1993). 

In addition, the Educational Testing Service found that among 270,000 test-takers in 1995 through 1997, college 
admissions test scores were highly correlated with initial teacher licensing scores (Praxis I and Praxis II), and the 
lowest average scores on both kinds of tests were those held by individuals who entered teaching without preparation 
(Gitomer, Latham, and Zimek, 1999). (Walsh describes this 14% of the sample as an "error" in the study since the 
individuals had not enrolled in a teacher education program; she misunderstands the fact that these Praxis test-takers 
were the entrants to teaching who used emergency or alternative routes. (Note 20) Prepared teachers scored much 
higher than unprepared teachers. 

While students who prepare to enter fields other than teaching have higher average test scores on measures like the 




SAT than do those preparing to enter elementary school teaching, there is no significant difference for prospective 
secondary teachers, most of whom earn a disciplinary degree along with their teaching certificate. The narrowing of 
this gap between prospective teachers and others is likely a function of the more rigorous admissions requirements for 
teacher education enacted in most states and the growth in wages between the early 1980s and the mid-1990s. 

Finally, the study found that graduates of NC ATE-accredited colleges of education passed the Praxis subject matter 
tests for teacher licensing at a significantly higher rate than did graduates of unaccredited programs, boosting their 
chances of passing the examination by nearly 10 percent (Gitomer, Latham, and Zimek, 1999). Walsh suggests that 
this higher Praxis pass rate might simply reflect the fact that NCATE schools could be located in states with low 
cutoff scores. However, additional analyses of the data by ETS and another independent study (Note 21) indicate that 
this is not the case. A more likely explanation is that NC ATE's requirements that colleges demonstrate how they 
screen applicants for general ability and that they ensure strong content backgrounds translate into somewhat greater 
attention to these matters in institutions that are accredited. These data suggest that standards may increase the general 
as well as specialized qualifications of prospective teachers. They do not suggest that removal of certification 
requirements brings higher ability individuals into teaching or keeps them there. 



It is important to recognize that labor market incentives operate among individuals actually entering teaching. For 
example, several studies of alternative certification programs found that the academic records of recruits varied 
substantially by teaching field, with alternatively-certified candidates in high demand shortage fields, such as 
mathematics and science, having much poorer academic records than candidates in other fields and than candidates 
from traditional teacher education programs in those same fields (see Natriello & Zumwalt, 1992, re: New Jersey; 

Lutz and Hutton, 1989 re: Dallas; Stoddart, 1992, re: Los Angeles). It is unlikely that eliminating requirements for 
training would increase the career attractions to teaching for academically able candidates as much as increased wages 
would. Meanwhile, eliminating training requirements could result in a less well-qualified teaching force, especially if 
the elimination of certification standards not only reduced the knowledge of entrants but also reduced pressures for 
competitive wages. 

The Private School Argument 

Finally, a claim sometimes made by opponents of teacher certification, including Walsh, is that private schools are 
more effective than public schools, and that this is because — or at least is not impeded by — the fact that private school 
teachers are not certified. There are two major problems with the private school "proof: First, there are conflicting 
findings about the relative effectiveness of public and private schools, with credible evidence on both sides of the 
question. Second, most private school teachers are certified and an even larger majority have specific preparation for 
teaching, even when they have not sought certification. 



On the effectiveness of private schools, Walsh cites Coleman, Hoffer, & Kilgore (1982), who examined data from the 
first wave of High school and Beyond surveys, conducted in 1980, and found evidence of higher performance for 
comparable students in Catholic and other private schools as compared to public schools. The researchers attributed 
their findings primarily to differences in student behavior across school sectors, measured by variables like lower 
rates of absenteeism, cutting class, and fighting, along with factors like more time spent on homework and higher 
individual student attendance. They also found that achievement was actually higher for comparable students who 
were in public schools that had these characteristics. Subsequent studies have produced findings that favor both public 
and private schools after controlling for student characteristics and school organization (Bryk & Lee, 1992; Lee & 
Bryk, 1988; Lee, Dedrick, & Smith, 1991). Most studies have pointed to variables like school and class size, school 
organization, and curriculum differentiation as critical variables in determining both public and private school 
effectiveness. When these factors are controlled, public school students often do as well or better than private school 
students in schools with similar features. 

Furthermore, differences in the preparation of public and private school personnel are not as large as many people 
assume. More than 30 states certify private school personnel (Feistritzer, 1984), and, when Coleman did his analysis, 
more than 85% of private and parochial school teachers were certified, as compared to about 95% of public school 
teachers (NCES, 1985). This has changed only slightly in the years since. Although certification is not required for 
private school teachers in all states, only 34% of private school teachers in 1993-94 (the most recent year for which 
national data are available), were not certified in their primary assignment field. Some of these teachers were certified 
in fields other than their primary assignment field. Many undertook teacher preparation, even though they did not 
apply maintain a state license or certificate. In 1993-94, public and private school teachers were almost equally likely 
to have received an undergraduate degree in education (68.9% for public vs. 61 .5% for private elementary teachers 
and 19.8% for public vs. 19.3% for private secondary teachers) (NCES, 1997, p. 25). The education degree as an 
indicator of preparation is quite partial, since the education degree has waned as certification increasingly requires a 
content degree with an education minor or credential. The percentage of 1992-93 bachelor's degree recipients who had 
taken education courses was 87. 1% for public school teachers and 71 .6% for private school teachers, (Note 22) and 
the average number of education credits earned was 37.4 for public school teachers as compared to 35.2 for private 
school teachers (NCES, 1997, table A-51). (Note 23) 

Public school teachers were also more likely to have taken subject matter degrees in their teaching fields than private 
school teachers. For example, 66% of public school mathematics teachers held a major or minor in the field, as 
compared to 58% of those in private school. (Goldhaber and Brewer, 2000 reported a similar finding.) The same 



differentials hold in other fields to somewhat lesser extents. The greater content preparation of public school teachers 
is likely a function of the fact that certification has required increasing amounts of subject matter coursework in the 
field to be taught, thus leveraging stronger content preparation for public school teachers in states where private 
school teachers are not required to hold certification. Almost all states now require certified teachers to hold at least a 
minor in the field to be taught, and many require a major in the field. 

Finally, even if it were true that untrained teachers were unusually effective in some private schools for students of 
comparable initial achievement levels — a point about which there is no published evidence — it would be a large leap 
of faith to assume that such teachers would be equally effective in schools where many students have much greater 
educational needs and students are not pre-selected for their academic ability, their positive school attendance and 
behavior, and their parents' income and interest in education. There are very large differences in the populations of 
students attending public and private schools in the United States, (Note 24) which have important implications for 
teachers' knowledge and skills. It is one thing for a teacher to offer information in whatever manner comes 
instinctively to students who are academically able, have learned to learn independently, and are well-supported at 
home by educated parents, tutors, and other supports for their learning. It is quite another thing to teach by the seat of 
the pants when students do not have these learning supports at home and may present a variety of language and 
learning differences. Being effective with students who need substantial support for their learning requires greater 
diagnostic ability and knowledge of how to present information and structure experiences in ways that help them 
become successful. Systematic knowledge about how to organize curriculum and reach students with special learning 
needs is most needed in the schools that serve most students with these needs. 



The remainder of Walsh's review continues the kind of misrepresentations documented above, appearing to rely on 
the belief that readers will read its accusations, but will not read or understand the research itself Although she 
prepared a draft appendix with 192 studies that sought to critique many of the studies she dismisses (often 
inaccurately), it was not published with the report. Appendix B, to which the reader is repeatedly referred for reviews, 
includes only 14 studies. Throughout the report, the reader is referred to this appendix for critiques of studies that do 
not appear there. The selection of research included in the published version of the report's appendix is very strange. 
Many strong studies — some of the key citations in the field — are omitted, along with the flawed rationales for 
dismissing them that now appear in a separately-published appendix. Some much less important and less well- 
designed studies are included, with the apparent goal of critiquing their size or designs as though they represented the 
dozens of studies not mentioned or excluded. Thus, the paper does not include information regarding most of the 
studies Walsh claims she has reviewed and does not provide evidence for her claim that, of all the studies cited in 
support of teacher education and certification, "none bear up to scrutiny," 

Here are just a few additional examples of major misrepresentations, 

1 . Goldhaber & Brewer (2000). In a string of citations, Walsh lists a study by Goldhaber and Brewer (2000), for 
its finding that teachers with a degree in their subject matter are more effective than those without such 
degrees. This study fits all of Walsh's desiderata: It is large (using a data set that includes more than 3,000 
teachers), recent, and published in a peer-reviewed journal. However, Walsh does not cite the authors' 
findings that certification status has an even greater influence on teachers' effectiveness than a degree in the 
subject area. Later, Walsh states, "...most research indicates that the most distinct problem in schools serving 
poor children is the number of teachers who are teaching subjects in which they have no expertise (Goldhaber 
& Brewer, 2000; ... Hawk, Coble, & Swanson, 1985). These studies do not show that certification status, as 
an isolated variable, has any significant effect on the achievement level of children who are poor or 
minority." (p. A6), Neither study examined the subject matter expertise of teachers in low-income schools, 
and both found strong effects of certification on student achievement. In fact, Goldhaber and Brewer wrote: 

Turning to an examination of the effect of teacher certification, we find that the type 
(standard, emergency, etc.) of certification a teacher holds is an important determinant of 
student outcomes. In mathematics, we find the students of teachers who are either not 
certified in their subject (in these data we cannot distinguish between no certification and 
certification out of subject area) or hold a private school certification do less well than 
students whose teachers hold a standard, probationary, or emergency certification in math. 

Roughly speaking, having a teacher with a standard certification in mathematics rather than a 
private school certification or a certification out of subject results in at least a 1 .3 point 
increase in the mathematics test. This is equivalent to about 10% of the standard deviation on 
the 12th grade test, a little more than the impact of having a teacher with a BA and MA in 
mathematics. Though the effects are not as strong in magnitude or statistical significance, the 
pattern of results in science mimics that in mathematics. Teachers who hold private school 
certification or are not certified in their subject area have a negative (though not statistically 
significant) impact on science test scores (p. 139), 

The authors note that the effect size of "having a teacher with a standard certification in mathematics rather 
than a private school certification or a certification out of subject" is "a little more than the impact of having a 
teacher with a BA and MA in mathematics," Of course, the certification itself includes requirements for 



Other Misrepresentations of Research Findings 




subject matter knowledge as well as for knowledge of teaching and learning. In fact, certified mathematics 
teachers are more likely to have a degree in the field than non-certified teachers. The fact that the study found 
a significant effect of certification status even after controlling for whether teachers had a degree in their field 
and after controlling for experience suggests that whatever is represented by the certification variable has an 
influence above and beyond the influence of content knowledge and classroom experience. 

2. Druva & Anderson (1983). This meta-analysis of 65 studies examined relationships between science teacher 
characteristics and teaching behaviors, student achievement in science, or both, using meta-analytic 
techniques to translate results from a wide range of studies into Pearson correlation coefficients in order to 
compare them. It found that ratings of teaching effectiveness by principals and students were most strongly 
correlated with the number of education courses taken, followed by student teaching grades, and teaching 
experience. On a teacher "effectiveness" scale composed of many teaching behaviors associated in process- 
product research with student achievement, both science training (examined in 28 studies) and education 
coursework and performance (examined in 47 studies) were related to effectiveness, as were teacher attitudes, 
values, and temperament. Associations with cognitive and affective student outcome measures were found for 
both science training and, to a somewhat smaller extent, for education coursework and performance, based on 
34 studies for each of these sets of variables. The authors concluded that: 

Student outcomes are positively associated with the preparation of the teacher, especially 
science training, but also preparation in education and academic work generally.... While the 
hiring official seeking a new science teacher certainly must look beyond information on the 
teacher characteristics considered in this study, information on some of these characteristics 
certainly is worthy of inclusion in the decision-making process.... In general, the hiring 
official would be well advised to employ teachers with thorough preparation in both 
professional education and the sciences being taught. There is a relationship between teacher 
preparation programs and what their graduates do as teachers (p. 477). 

Walsh seeks to dismiss the results of this study in part by misreporting them. She states the study "did not 
show the benefit of education coursework on student achievement" (p. 19), and that education coursework is 
not significantly related to student outcomes, although significance statistics were not reported in the study. 
This assertion is not supported by the authors' reported findings that both science coursework and education 
training showed a relationship to teacher effectiveness as defined by student outcomes (in both cases, though 
to a greater extent for science coursework) (Note 25) as well as teaching behaviors and ratings (reported in 
the case of education coursework only). 

3. Darling-Hammond (2000). Walsh criticizes and misquotes a study that this author conducted, which 
examined both the literature on teacher characteristics and student achievement and conducted a regression . 
analysis of state-level data from the National Assessment of Educational Progress and the Schools and 
Staffing Surveys (Darling-Hammond, 2000). The study found that measures of teacher preparation and 
certification were by far the strongest correlates of student achievement in reading and mathematics, both 
before and after controlling for student poverty and language status. The conclusion discussed a number of 
potential reasons for these large effects: 

The strength of the "well-qualified teacher" variable may be partly due to the fact that it is a 
proxy for both strong disciplinary knowledge (a major in the field taught) and substantial 
knowledge of education (full certification). If the two kinds of knowledge are interdependent 
as suggested in much of the literature, it makes sense that this variable would be more 
powerful than either subject matter knowledge or teaching knowledge alone. It is also 
possible that this variable captures other features of the state policy environment including 
general investments in, and commitment to, education, as well as aspects of the regulatory 
system for education, such as the extent to which standards are rigorous and the extent to 
which they are enforced.... Finally, there may be unmeasured correlations between the extent 
to which states enact and enforce high standards for teachers and the extent to which they 
have enacted other policies that are supportive of public schools. Although it does not appear 
that teaching standards are strongly related to investments regarding class sizes or to overall 
education spending, it is possible that there are other factors influencing student achievement 
which generally co-exist with teacher quality and which were unmeasured in these estimates. 

Walsh seeks to invalidate these findings by raising two complaints, one of which is inaccurate and the other 
of which is a matter of legitimate discussion in the field. She states, incorrectly, that, "Darling-Hammond did 
not control for class size differences among the states" (p. 26). State-level differences in average class size 
were in fact included in the analyses, and the variable had a very small, insignificant effect. Walsh also 
complains that the state-level analyses suffer from aggregation bias because they used average student test 
scores — a critique she also levels against other studies she cited approvingly for their findings in other parts 
of the paper (see e.g. Ferguson, 1991; Strauss & Sawyer, 1986; Coleman, 1966). (Note 26) There are 
legitimate debates in the field on this point, and I addressed this question in the study itself, as I do again 
below in the section on "Methodological Issues." For purposes of tracking broad policy trends at the state 
level, analyses of state level data offer one useful lens. This perspective was shared by the nine reviewers 
who recommended this paper's publication in a peer-reviewed journal and a peer-reviewed research report 




series. 



Finally, the literature review contained in this study is repeatedly mischaracterized throughout Walsh's paper 
and her appendix as minimizing or ignoring the influences of verbal ability and subject matter preparation for 
teaching. 



On the relationship between academic ability and teacher effectiveness, Walsh states: 



Darling-Hammond (1999, p. 6) claims there is "little or no relationship between teachers’ 
measured intelligence and their students’ achievement.’’ She supports this statement with two 
studies by Soar, Medley and Cocker (sic) (1983) and Schalock (1979). These two studies 
simply recycle research from the 1940s and earlier, none of which is retrievable for scrutiny 

(p.21). 



Walsh misrepresents this analysis by quoting a portion of a sentence out of context and citing the reviews that 
summarized research on IQ tests as an example of the inappropriate use of older studies. Here is what I 
actually said; 



While studies as long ago as the 1940s have found positive correlations between teaching 
performance and measures of teachers’ intelligence (usually measured by IQ) or general 
academic ability (Hellfritsch, 1945; LaDuke, 1945; Rostker, 1945; Skinner, 1947), most 
relationships are small and statistically insignificant. Two reviews of such studies concluded 
that there is little or no relationship between teachers’ measured intelligence and their 
students’ achievement (Schalock, 1979; Soar, Medley, & Coker, 1983). Explanations for the 
lack of strong relationship between measures of IQ and teacher effectiveness have included 
the lack of variability among teachers in this measure and its tenuous relationship to actual 
performance (Vernon, 1965; Mumane, 1985). However, other studies have suggested that 
teachers’ verbal ability is related to student achievement (e.g., Bowles & Levin, 1968; 
Coleman et al., 1966; Hanushek, 1971), and that this relationship may be differentially strong 
for teachers of different types of students (Summers & Wolfe, 1975). Verbal ability, it is 
hypothesized, may be a more sensitive measure of teachers’ abilities to convey ideas in clear 
and convincing ways (Mumane, 1985).’’ 



Walsh’s attempt to distort the text misses two critical points: First, studies of the relationship between IQ and 
teaching effectiveness (which I noted had found positive though small relationships) were primarily 
conducted before the 1960s, because IQ tests came into question as measures of ability at that time and were 
no longer often available in large data sets thereafter. Measures of verbal ability became more popular and 
widely available in data sets in the 1960s and following, and showed somewhat stronger relationships with 
teacher outcomes, as 1 reported in my summary. The studies 1 cited include many of the same ones that Walsh 
cites for this proposition — a point she does not acknowledge as she tries to suggest, inaccurately, that I 
minimize the value of measures of academic ability for teachers. (Note 27) 



On the topic of subject matter knowledge, Walsh also suggests on numerous occasions that I seek to 
minimize the importance of teachers’ knowledge of content. She offers my work as an example of her 
sweeping statement that ’’certification advocates ... offer evidence that knowledge of subject matter has little 
effect on teaching performance’’ (p. 19). Here is what 1 actually said in my brief summary of the literature, 
offering an analysis that clearly acknowledges the importance of subject matter knowledge for teaching and 
interprets the mixed results of studies in terms of what teachers may need to know in order to teach different 
things. 



Byrne (1983) summarized the results of thirty studies relating teachers’ subject matter 
knowledge to student achievement. The teacher knowledge measures were either a subject 
knowledge test (standardized or researcher-constructed) or number of college courses taken 
within the subject area. The results of these studies were mixed, with 1 7 showing a positive 
relationship and 14 showing no relationship. However, many of the "no relationship" studies, 
Byrne noted, had so little variability in the teacher knowledge measure that insignificant 
findings were almost inevitable. Ashton and Crocker (1987) found only 5 of 14 studies they 
reviewed exhibited a positive relationship between measures of subject matter knowledge 
and teacher performance. 

It may be that these results are mixed because subject matter knowledge is a positive 
influence up to some level of basic competence in the subject but is less important thereafter. 
For example, a controlled study of middle school mathematics teachers, matched by years of 
experience and school setting, found that students of fully certified mathematics teachers 
experienced significantly larger gains in achievement than those taught by teachers not 
certified in mathematibs. the differences in student gains were greater for algebra classes 
than general mathematics (Hawk, Coble, & Swanson, 1985). However, Begle and Geeslin 



(1972) found in a review of mathematics teaching that the absolute number of course credits 
in mathematics was not linearly related to teacher performance. 

It makes sense that knowledge of the material to be taught is essential to good teaching, but 
also that returns to subject matter expertise would grow smaller beyond some minimal 
essential level which exceeds the demands of the curriculum being taught. This interpretation 
is supported by Monk’s (1994) more recent study of mathematics and science achievement. 
Using data on 2,829 students from the Longitudinal Study of American Youth, Monk (1994) 
found that teachers’ content preparation, as measured by course work in the subject field, is 
positively related to student achievement in mathematics and science but that the relationship 
is curvilinear, with diminishing returns to student achievement of teachers’ subject matter 
courses above a threshold level (e.g., five courses in mathematics). 

It may also be that the measure of subject matter knowledge makes a difference in the 
findings. Measures of course-taking in a subject area have more frequently been found to be 
related to teacher performance than have scores on tests of subject matter knowledge. This 
might be because tests necessarily capture a narrower slice of any domain. Furthermore, in 
the United States, most teacher tests have used multiple-choice measures that are not very 
useful for assessing teachers’ ability to analyze and apply knowledge. More authentic 
measures may capture more of the influence of subject matter knowledge on student 
learning. For example, a test of French language teachers' speaking skill was found to have 
significant correlation to students' achievement in speaking and listening (Carroll, 1975). 

It seems logical that teachers' abilities to handle the complex tasks of teaching for higher- 
level learning are likely to be associated, to varying extents, with each of the variables 
reviewed above: verbal ability, adaptability and creativity, subject matter knowledge, 
understanding of teaching and learning, specific teaching skills, and experience in the 
classroom, as well as interactions among these variables. In addition, considerations of fit 
between the teaching assignment and the teacher’s knowledge and experience are likely to 
influence teachers’ effectiveness (Little, 1999), as are conditions that support teachers' 
individual teaching and the additive effect of teaching across classrooms, such as class sizes 
and pupil loads, planning time, opportunities to plan and problem solve with colleagues, and 
curricular supports including appropriate materials and equipment (Darling-Hammond, 
1997). 



Finally, Walsh suggests in several places that I have characterized the research as indicating a "negative 
relationship between student outcomes and the NTE subject matter tests" (p. 19). In fact, I stated that "Studies 
of teachers' scores on the subject matter tests of the National Teacher Examinations (NTE) have found no 
consistent relationship between this measure of subject matter knowledge and teacher performance as 
measured by student outcomes or supervisory ratings. Most studies show small, statistically insignificant 
relationships, both positive and negative (Andrews, Blackmon & Mackey, 1980; Ayers & Qualls, 1979; 
Haney, Madaus, & Kreitzer, 1986; Quirk, Witten, & Weinberg, 1973; Summers & Wolfe, 1975)." (Note 28) 
Walsh misrepresents this statement numerous times. 

Methodological Issues 

One of the ways that Walsh seeks to make much of the research on teacher education disappear is by suggesting that it 
is inappropriate to cite studies that are older, smaller, use measures of performance other than student achievement 
scores, are aggregated at a level above the classroom, or are published in venues other than peer-reviewed journals. 



As noted above, Walsh uses a double standard in selecting research to reject when it finds evidence of the influence of 
teacher education on student learning and research to cite for her own purposes. While she discounts the findings of 
many dissertation studies and technical reports because they were not published in peer-reviewed journals, in making 
her own claims, she cites at least 1 5 studies that were not published in peer-reviewed journals or technical report 
series and at least 20 that were published before 1 980, including some that she elsewhere dismissed from 
consideration because she did not like specific findings. For findings she likes, she also cites several that use 
supervisory ratings as the only measures of teacher effectiveness and others that she later dismisses for aggregation 
bias. Sometimes she represents the studies’ findings accurately; sometimes not. Many of the studies she cites for 
various propositions do not contain the findings for which they are cited — or, in several cases, any data on the 
question at all. 



I would not argue, as Walsh does, that none of these studies have value as contributions to the literature. However, the 
double standard she applies in using studies of different eras, sizes, aggregation levels, dependent variables, and 
publication statuses perhaps proves the point that to evaluate the weight of evidence in a field it is often necessary to 
triangulate findings that used different methods, over different time periods, and at different levels of aggregation to 
see where there is an accrual of evidence over time and across methods. Of course it is important to do this with 
appropriate attention to the methodological strengths and weaknesses of various studies and lines of research. 
Unfortunately, Walsh often does this poorly, appearing to misunderstand critical research design issues. Below, I 
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discuss the issues of study size and design, level of aggregation, choice of dependent variable (including the use of 
supervisory ratings of teacher performance), age, and venue of publication. 

Study Size and Design 

In one part of her review, Walsh bemoans the lack of experimental research. She then rejects the results of studies 
with experimental designs because of their smaller sample sizes and cites almost exclusively non-experimental 
correlational studies, which — though larger — lack direct controls for the variables of interest and must rely on 
statistical manipulations of data to account, indirectly, for these other influences. This kind of correlational research 
is, of course, legitimate for staking out broad possibilities in relationships among variables, but it has its own 
limitations. Many of the more carefully controlled experimental designs can in fact offer more solid evidence about 
effects, because the ’’treatment” they are studying is known and the samples can be better controlled than is true for 
large correlational studies that use proxies and statistical controls rather than direct observation of the phenomena of 
interest. Medical research, for example, typically uses small sample experimental research as the basis for establishing 
the possibilities of effects, while using large correlational studies as rough indicators of possible relationships that 
then require further examination. Single case studies of clinical findings are part of the medical research base along 
with small experiments sometimes carefully controlled and sometimes not, larger clinical trials, and correlational 
studies looking at broad tendencies. 



The usefulness of small, experimental and quasi-experimental studies — including those that Walsh cites and 
sometimes dismisses (and other times embraces, depending on her reading of and agreement with the findings) — is 
not in the definitiveness of their individual findings but in their contribution to a larger body of work from which a 
preponderance of evidence can be examined. Although medical researchers generally consider correlational studies to 
comprise a weaker source of evidence about effects than smaller experimental designs, they recognize that mixed 
methods of research serve complementary purposes. 



Of course, one of the reasons correlational studies must be interpreted with caution is that there is always the question 
of what direction the correlations may point, sometimes referred to as "reverse causation.” There is also the problem 
that variables in these studies are frequently crude proxies for the actual measures of interest and may either fail to 
capture the intended construct or in fact be reflecting the influences of other unmeasured variables. As noted above, 
many of the variables that can arguably be said to reflect constructs of interest are highly correlated with one another. 
Furthermore, many of the variables of interest are not well-represented in large data sets. Thus it is critical to 
represent in any review of research a range of studies that can tease apart the different relationships of interest with a 
range of measures. 

Level of Aggregation 

Another criticism used to dismiss some studies’ findings as irrelevant is the charge of "aggregation bias.” For 
example, Walsh dismisses studies that include favorable findings about the value of teacher education in which data 
are aggregated at the level of the school or district, although she, herself, cites similarly aggregated data for her 
conclusion that verbal ability matters most (e.g. Coleman, 1966; Ferguson, 1991; Strauss & Sawyer, 1986). More 
important, this critique misses a crucial point about how research results accrue and are triangulated to look at 
possible relationships among conditions and outcomes. Just as individual level data about health practices and 
outcomes inform medical research, for example, so do highly aggregated data at the level of cities, counties, and even 
countries when researchers seek to understand why, for example, women in some nations have low levels of breast 
cancer or men have low levels of heart disease. Studies at different levels of aggregation provide different kinds of 
insights about the phenomena under study. In building a corpus of research on any topic, a wide array of research 
strategies and levels of analyses are used. 

It is true that the size of measured effects of different variables can vary at different levels of the system; however, it 
is not always clear in which way the bias will operate. Often, the general direction of the results holds at different 
levels of the system, even if effect sizes differ. For example, in their Alabama study, Ferguson and Ladd (1996) found 
the effects on student achievement of teachers' test scores, masters degrees, and experience held at both the district 
and school levels in terms of both significance and directionality. There are pros and cons of both kinds of analyses. 
On the one hand, disaggregated data can exhibit greater measurement error. On the other hand some analysts have 
argued that omitted variables may bias the coefficients of school input variables upward when data are aggregated to 
the district or state level (Hanushek, Rivkin, & Taylor, 1995). However, this generalization does not always prove 
true. For example, although Summers and Wolfe (1975) found that selectivity ratings of each teacher's undergraduate 
institution were important in explaining 6th grade students' achievement when examined at the individual teacher 
level, this relationship disappeared with they aggregated the college ratings and other school inputs into school-level 
averages. This contradicts the assumption about the usual direction of aggregation bias. 



Of course, omitted variables can bias results at any level of the system. Sometimes, especially when the goal of a 
study is to evaluate broad trends and policy influences, it is important to have data aggregated and analyzed at 
multiple levels. For interpreting the weight of evidence on a particular issue, the most important question is whether 
consistent results are found at different levels of aggregation. Just as Walsh cites highly aggregated data as well as 
less aggregated data on the question of the influences of verbal ability, so the studies examined here reveal influences 
of measures of teacher education and certification on student achievement at the levels of state (Darling-Hammond, 
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2000c), school district (Ferguson, 1991; Ferguson & Ladd, 1996; Strauss & Sawyer, 1986), school (Ferguson & Ladd, 
1996; Fetler, 1999), and individual teacher (Goldhaber & Brewer, 2000; Hawk, Coble, & Swanson, 1985; Monk, 
1994). 



Measures for Assessing Teacher Performance 

Walsh argues that studies using various ratings of student performance other than student achievement test scores 
should be discounted, noting that supervisory ratings '’can be too subjective to measure teacher quality accurately” (p. 
20). As support for this, she cites in her appendix a review of research on teacher evaluation I conducted with 
colleagues at the RAND Corporation (Darling-Hammond, Wise, & Pease, 1983). While her statement of why I cited 
the review in another article is completely inaccurate, (Note 29) she is correct when she notes that teacher evaluations 
by principals and other school-based supervisors have been found to lack strong reliability. Our study of evaluation 
practices noted that this has been a function of principals’ lack of time, inadequate expertise for evaluating all teaching 
situations, insufficient evaluation training, and inappropriate instrumentation. However, this critique does not extend 
to ratings of performance that are based on structured observations conducted by trained, expert raters that have been 
developed and demonstrated to have high reliability. Some of the studies Walsh dismisses use systematic ratings 
systems by trained observers (e.g. Ferguson & Womack, 1993; Guyton & Farokhi, 1987). The extent to which ratings 
of performance should be considered or discounted depends on who conducts the rating process, with what training 
and instrumentation, under what conditions, and with what efforts to enhance reliability. 

Age of Studies 

The age of studies is also a legitimate but not determinative issue. Studies do not become invalid merely because they 
are old. While Walsh argues that many older studies using large data sets lacked certain kinds of variables as controls, 
this does not stop her from citing many of these studies for propositions with which she agrees. More important, the 
designs of some older studies are at least as strong as some of the more recent studies, and weak studies exist now as 
then. There is not a strong relationship between study vintage and quality. It is certainly true that teacher education 
programs and certification requirements have changed over time, so that inferences from studies conducted in one era 
do not automatically generalize to others; the extent to which one can learn something of use from a study depends on 
how well the variables are defined and on a knowledge of their relevance to more recent conditions as well as on the 
strengths and limits of its methodology. 

Vintage does influence the prevalence of studies of certain kinds. With respect to studies of the effects of teacher 
education and certification, a large number of studies were conducted in the high-demand era of the 1960s and ’70s 
when there was great variability in entry pathways and much interest in the topic. It is also true that federal funding 
for educational research was substantially larger before 1980 than it was during the severe budget cuts of that decade. 
In addition, in times of relatively low demand, like most of the 1980s, virtually all teachers were certified and there 
was too little variability to find effects of this variable in large-scale studies. Few studies were concerned with these 
issues and few data sets had measures of teacher education variables. Interest and data on this topic have just begun to 
return in the 1990s. Those who are interested in the extent to which — and the ways in which — different kinds of 
preparation may matter for teacher performance and student learning can and should be informed by earlier studies 
where they are applicable to the questions under study. 

Publication Venue 

Although Walsh is incorrect in her statement that dissertations are not retrievable (there are library systems for doing 
so, if sometimes less than convenient), it is legitimate to suggest that the kind of review they have received is often 
more variable, and may be less strenuous depending on the university and department, than for many peer-reviewed 
journals. There are certainly some universities whose dissertation review process is more rigorous than some journals, 
but the reverse is also certainly true. The same variability in review stringency is true for conference papers and 
technical reports. However, Walsh herself cites a substantial number of unreviewed papers in support of various 
positions she takes. There are different schools of thought about how to treat these papers in reviews. Some would 
argue, as does Veenman (1984), a reviewer cited by Walsh, that the use of all identified studies is justified for a 
review that seeks to delineate global trends where large numbers of findings are similar (p. 166). Others would argue 
that papers that have not been published with peer review should be used only when the review includes a critique of 
each study's methods. Still others might argue, as Walsh does (at least rhetorically if not in practice), that such studies 
should be excluded from consideration. I accept the point that it is a useful common ground to rely on research 
published in peer-reviewed journals, and I restrict the analysis in this paper to those studies. Even with this criterion, 
there is substantial evidence to be weighed and discussed. 

Who is Affected by this Debate? 

The critical issue here is not the protection of researchers’ reputations or the turf of schools of education but the 
protection of students, especially low-income students and students of color who are disproportionately taught by 
unprepared and uncertified teachers. As Walsh’s paper shows in her references to data on the disparities in access to 
qualified teachers for students in Baltimore, the children most affected by these arguments are economically and 
educationally disadvantaged children in central cities who are substantially abandoned by the funding and hiring 
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protections that should operate to provide a foundation for their education. These are the students whose education is 
most undermined by their lack of access to teachers who have the knowledge and skills to ensure that they learn to the 
new high standards the society and the state demand. 

What the statistics on the lack of certified teachers actually mean on the ground is that many of Baltimore’s most 
educationally vulnerable children — most of them African American — are taught in their elementary school years by 
teachers who have had no training in how to teach them to read, much less to develop other basic and higher order 
skills they must have to succeed in school and life. When they fail to learn, they begin the tortuous process of 
educational failure that will end for many of them in dropping out or being unable to pass the state tests that would 
grant them a diploma. This then launches a life spent either in a marginal part of the economy that barely yields 
subsistence wages or, as is true for more than 50% of high school dropouts, in the inability to gain any job at all. In 
today’s economy, these young people are fated to become part of the growing criminal justice system, as incarceration 
is increasingly linked to inadequate education. More than half of the growing number of inmates in the United States 
are functionally illiterate and cannot gain access to today’s labor market. This is not unrelated to the fact that so many 
low-income students have been taught by teachers who never learned how to teach them to read. 



Illogical Policy Conclusions 

The disparities in access to qualified teachers in Maryland are a function of a state school finance system that has 
underfunded Baltimore’s schools for decades, along with inadequate incentives — for example, service scholarships, 
forgivable loans, and recruitment attractions like salaries and housing assistance — to encourage individuals to acquire 
strong training and then teach in high-need fields and locations. The Abell Foundation report does not argue for more 
equitable funding for the schools that serve Maryland’s poor and minority students or for stronger incentives to attract 
well-prepared teachers to these schools. In fact, the report cites approvingly a paper prepared to stave off an equity 
lawsuit in Maryland (Hanushek, 1996b) which argues against district investments in smaller class sizes or higher 
salaries in Baltimore, asserting that ’’Baltimore City would not benefit from additional resources as much as it could 
benefit by better school management.” (Note 30) The Abell Foundation report argues that the enormous disparities in 
resources and qualified teachers between Baltimore and other districts are not a problem because teacher certification 
does not mean anything, and that in fact the solution is to do away with certification altogether. 



In suggesting that devolving all hiring decisions to principals is the answer to the problem of recruitment for the 
schools serving minority and poor children, Walsh ignores the fact that, even if all principals had infinite information 
at their disposal about the likely effectiveness of teachers and made wise, fully informed choices (two assumptions 
that have been challenged by some research on teacher selection practices), principals do not control the major levers 
for addressing the problems of unequal supply: unequal district revenues, noncompetitive teacher salary levels, and 
the policies that govern recruitment and preparation that would allow them to seek out and hire the individuals they 
might most want to recruit. 



Eliminating certification requirements would eliminate pressures for competitive wages or recruitment incentives for 
teachers, since an open marketplace in a resource-constrained public sector could resolve shortages by lowering 
standards. In addition, eliminating certification requirements would eliminate evidence about disparities in students’ 
opportunities to learn, for if there are no minimum standards, there will be no evidence of differences in the extent to 
which they have been achieved by teachers working with different groups of students. This would in turn reduce 
pressures for the creation of policies to rectify these inequities. Finally, eliminating such standards would remove the 
mechanisms states have been developing and improving to be sure that teachers know their content well, know how to 
teach the content to students, know how to teach fundamental skills like reading, and have the ability to meet the 
special needs of learners who may have learning disabilities that require distinct teaching strategies, whose first 
language is not English, or who simply struggle with certain kinds of academic tasks and need diagnostic assistance. 



The outcome of Walsh’s argument, were it to be successful in the policy community, would be continued inequality in 
funding, depressed salaries for teaching in high-need areas, continued lack of access for poor children to a stable 
teaching force of well-qualified teachers by any definition, and tragic loss of a productive future for students who are 
underserved. 



To be sure, certification is but a proxy for the subject matter knowledge and knowledge of teaching and learning 
embodied in various kinds of coursework and in the evidence of ability to practice contained in supervised student 
teaching. It is true that certification is a relatively crude measure of teachers’ knowledge and skills, since the standards 
for subject matter and teaching knowledge embedded in certification have varied across states and over time, are 
differently measured, and are differently enforced from place to place. The quality of preparation in both university 
programs and other alternatives has varied as well, although a number of states have made substantial recent headway 
in strengthening teachers’ preparation and reducing this variability. Given the crudeness of the measure, it is perhaps 
remarkable that so many studies have found significant effects of teacher certification. 



This does not mean that we should be sanguine about certification policies. There are questions about the quality of 
tests, courses, and institutions that are the subject of study and action across the country (see, for example, Darling- 
Hammond, Wise, & Klein, 1 999), The answer to flaws that may be perceived, however, is not to eliminate or 
undermine the pathways that enable and require teachers to gain knowledge and students to have access to teachers 
who have the knowledge they need. If teacher knowledge and skill about both content and how to teach it is 
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important, as substantia] evidence suggests it is, the most sensible policy goal is to work to improve preparation 
opportunities and certification standards so that they increasingly approximate what teachers need to know and do in 
order to be successful with diverse students. 

As Levin (1980) noted, certification is a critically important exercise in the economics of information that should be a 
target of continual improvement: 

(T)he facts that we expect the schools to provide benefits to society that go beyond the sum of those 
conferred upon individual students, that it is difficult for many students and their parents to judge 
certain aspects of teacher proficiency, and that teachers cannot be instantaneously dismissed, mean 
that somehow the state must be concerned about the quality of teaching. It cannot be left only to the 
individual judgments of students and their parents or the educational administrators who are vested 
with managing the schools in behalf of society. The purpose of certification of teachers and 
accreditation of the programs in which they received their training is to provide information on 
whether teachers possess the minimum proficiencies that are required from the teaching function. 

Because this is an exercise in the provision of information, it is important to review the criteria for 
setting out how one selects the information that is necessary to make a certification or accreditation 
decision (p, 7), 



Conclusion 

Kate Walsh has dismissed or misreported much of the existing evidence base in order to argue that teacher education 
makes no difference to teacher performance or student learning and that students would be better off without state 
efforts to regulate entry into teaching or to ensure certain kinds of teachers' learning. While she argues for recruiting 
bright people into teaching (and who could disagree with that?), her proposals offer no incentives for attracting 
individuals into teaching other than the removal of preparation requirements. While this proposal is couched as the 
elimination of "barriers" to teaching, evidence suggests that lack of preparation actually contributes to high attrition 
rates and thereby becomes a disincentive to long-term teaching commitments and to the creation of a stable, high 
ability teaching force. Lack of preparation also contributes to lower levels of learning, especially for those students 
who most need skillful teaching in order to succeed. 

The evidence from research presented here and elsewhere makes clear that the policies Walsh endorses could bring 
harm to many children, especially those who are already least well served by the current system. Those who make 
such arguments for eliminating one of the few protections these children have should bear the burden of proof for 
showing how what they propose could lead to greater equity and excellence in American schools. 

Notes 

1. The research assistance of Lisa Marie Carlson is gratefully acknowledged. 



2, "Teacher Certification Reconsidered: Stumbling for Quality" is published through the Abell Foundation website: 
www,abel lfoundation.org. The version of the report that was publicized and published on this website in October, 

2001 is the basis for this response. The report has since been amended. In a reply to my response posted to the Abell 
Foundation website, Walsh noted that some of the errors I pointed out have been removed in the hard copy version the 
foundation published in December 2001, 

3, In addition to the Abell Foundation, these include the Fordham Foundation, which has issued a "manifesto" urging 
the elimination of teacher education and certification requirements. 

4, See The Research and Rhetoric on Teacher Certification: A Response to Teacher Certification Reconsidered, at 
http://www,nctaforg. 



5. See Teacher Certification Reconsidered: Stumbling for Quality, A Rejoinder (November, 2001) at 
www.abellfoundation.org. 

6. A separate appendix is published on the Abell Foundation website, Soem of its entries have changed as criticisms 
of the report have been lodged, 

7. See, for example, footnote 1 8 on p, 13 where Walsh refers readers to Appendix B for analysis of six studies, only 
two of which (Guyton & Farokhi, 1987; Monk, 1994) are actually included there. Appendix B of the published 
version of Walsh's report includes only 14 of 192 studies originally included in her draft of July 23, 2001 and does not 
include most of the key studies on the topic, A longer appendix was later added to the Abell Foundation website. 
Readers who consult with that document will find that many of the studies listed are not concerned with teacher 
education but are cited for other reasons related to one of Walsh's own arguments; many others are not reviewed 
because they were not retrieved or were deemed too old or too small; still others are "reviewed" only in the sense that 



complaints are made about them or about the way they were cited by another researcher. 

S. In a reply to my response, Walsh and Podgursky (2001) suggest that Wenglinsky referred only to in-service 
education. However, the NAEP questions Wenglinsky analyzed for evidence of teacher learning covered coursework 
or professional development teachers had encountered before or after entering teaching. The stem for these questions 
was in each case one of the following: "During the past five years, have you taken courses or participated in 
professional development activities in any of the following?" or "Have you ever received training in any of the 
following, either in courses or in-service education?" 

9. Another study by the California Commission on Teacher Credentialing found the attrition rates of Los Angeles 
Teacher Trainees who dropped out before they entered teaching to be quite high. Of the first cohort, 80.3% completed 
the first year of training and only 64.6% completed the second year and received a clear credential the year after 
(Wright, McKibbon, and Walton, 1987). This 35% attrition rate prior to graduation from the program added to the 
53% attrition rate of those who completed the program but left the district within the subsequent 7 years (Stoddart, 
1992) left only about 30% of the original cohort in the district after 7 years. 

10. In her Education Next article, Walsh (2002) lists a set of studies with sample sizes of up to 55 teachers as "too 
small to produce results that are reliable or that can be generalized to the larger population," (on-line version, p. 9). 
However, in her reply to me (Walsh and Podgursky, 2001 , p. 14), she states that because Miller, McKenna, & 
McKenna’s study was a matched pair study, a "gold standard of research," its small numbers (18 teachers for 
examining student achievement effects) are justified. Yet just pages earlier in the same document (p. 8), she and 
Podgursky criticize another matched pair study (Hawk, Coble, & Swanson, 1985) which has a larger sample (36 
teachers) and stronger design for evaluating student achievement (Miller et al. drop most of their teachers and the 
matched comparison design when they evaluate student test scores) as lacking statistical controls (also missing in the 
Miller et al. study) and failing to adjust for pre-test scores of students (Miller, McKenna and McKenna do not even 
present the pre-test scores of students). The Hawk et al. study, which Walsh originally cited approvingly as an 
argument for content knowledge is now dismissed by Podgursky as "small and not well-controlled" to avoid having to 
acknowledge its results, which find positive effects of teacher certification on student achievement. 

1 1 . Personal communications with economist Susanna Loeb and statistician William Billet. 

1 2. As one of dozens of examples of general sloppiness, neither the Goldhaber and Brewer study nor the Hawk, 

Coble, and Swanson study cited by Walsh for this proposition even treated the question of whether "the most distinct 
problem in schools serving poor children is the number of teachers who are teaching subjects in which they have no 
expertise." Neither study examined or reported on the socioeconomic status of students or the distribution of teachers 
in schools serving different children. 

13. As the study clearly states, California uses emergency permits for those who lack either subject matter 
competence or pedagogy or both. The requirement for a clear credential is passage of both subject matter competence 
and a set of pedagogical requirements, whether these are completed in a "traditional" or an "alternative" program, 
which in California would be an internship model requiring the candidates to meet the same standards as traditional 
programs. In fact, the composition of the emergency permit pool in California is nearly the opposite of what Walsh 
seems to surmise. This pool includes many teachers who have passed the subject matter test (or alternative content 
course requirements) in mathematics but who have not completed teacher education requirements. It also includes 
many teachers who have passed a basic skills test but have not completed either the subject matter or teacher 
education requirements for a clear credential. It includes very few individuals who have completed teacher education 
requirements but who have not completed subject matter requirements, since demonstration of subject matter 
competence is a prerequisite for entering the student teaching or internship portion of teacher education in California. 
Furthermore, experienced teachers who may be teaching math out of field would generally have been included in 
Fetler’s data set as credentialed, since out of field teaching is not monitored by the state through the data set he used. 

14. The original appendix was included in Walsh’s draft dated July 23, 2001 . Her final complete appendix published 
in October, 2001 modifies this statement only slightly, stating, "The author’s principal and clear lament is the lack of 
subject matter knowledge in mathematics, with little mention at all of education coursework that may be lacking." 

15. High-minority schools were defined as those with more than 90% students of color; low-minority schools had 
fewer than 30%. High-achieving schools were defined as those in the top quartile of achievement on the SAT-9 tests 
used by the state; low-achieving schools were those in the bottom quartile. 

16. Rosalind Rossi, "Teacher woes worst in poor schools," Chicago Sun Times, October 10, 2001. 



17. Walsh states that, "L. Darling-Hammond ... presents a chart using an ambiguous term ’Teacher Qualifications' 
which accounted for nearly half of the student achievement gains." (p. 17). The chart to which Walsh alludes actually 
referred to another study by Ferguson (1996) and was clearly labeled as such. Another chart next to this one was 
drawn directly from a table in the Greenwald, Hedges, and Laine study, and was also clearly marked. 



18. In a later response to my reply (Walsh & Podgursky, 2001), Walsh notes that she cited Ferguson & Womack in 



error and meant to cite Ferguson and Ladd (1996). However, this study is one she should have discounted due to its 
level of aggregation if she were adhering to her own standards for evaluating research. 

19. One odd criticism is that the institution, Arkansas Tech, has "low entrance requirements, making it unlikely that 
enough variance in student ability, background and coursework is present to reflect a broader population. The 
variance may be too narrow or at least skewed." Walsh seems to be unaware that the variance in student ability 
measures is usually much larger in large state universities like this one than it is in more selective colleges, thus 
making some kinds of inferences more, rather than less supportable. The more appropriate question about single 
institution studies is whether they may generalize to unlike institutions, a legitimate point that Walsh does not raise, 
and that should be answered by conducting studies within and across institutional contexts. 

20. Some may also have been those teachers who needed to take the Praxis as an entrance examination for a post- 
baccalaureate teacher education program. 

21. The ETS re-analysis is soon to be published. An earlier analysis of the federal Baccalaureate and Beyond data 
base found that 1993 graduates of NCATE-accredited teacher education programs were about 50% more likely to 
have scored above the 50th percentile on SAT and ACT tests than graduates of non-NCATE teacher education 
programs (Shotel, 1998). NCATE graduates had also taken more social science, computer science, advanced foreign 
language credit, pre-college mathematics, and teaching coursework and fewer remedial English courses than non- 
NCATE graduates, with other areas being approximately equal (Shotel, 1998). 

22. The proportions who had taken other kinds of liberal arts coursework also differed little. For example, the 
proportion of 1 992-93 bachelor's degree recipients who had taken college coursework in mathematics at the level of 
calculus and above was 18.3% in public schools and 16.9% in private schools; science was 77.2% vs. 73.5% (table A- 
51). 



23. These statistics pertain to the youngest teachers in public and private schools: 1992-93 bachelors degree recipients 
hired by 1993-94. These teachers are the least likely to be certified, even though they have taken education 
coursework at rates nearly as high as public school teachers. This suggests that many of these teachers may have 
prepared to teach but did not seek or secure state certification. In 1993-94, NCES reports that about 36% of private 
school teachers held no certificate in their primary assignment field (the data are not presented regarding their 
certification in another field other than the primary teaching assignment). The rates of non-certification ranged from 
27% for those with 20 or more years of teaching experience to 51% for those with 3 or fewer years of teaching 
experience (NCES, 1997, table A3. 14a). 

24. For example, while most private school students (52%) attend schools that are less than 10% minority, only 31% 
of public school students do (NCES, Digest of Education Statistics, 1999, p. 71, table 60 and p. 119, table 99). 

African American and Latino students are at least 50% more likely to attend public than private schools. (NCES, 

1997, Table A2.13). Most low-income students and students of color now attend public schools in urban public school 
districts. 

25. Walsh objects to a composite "education and performance" variable created by the authors, which included the 
amount of education coursework, student teaching grade, GPA, and science teaching experience. 



26. In Walsh's original appendix, this study is further critiqued because the reviewer was not clear on the meaning of 
the term "out-of-field" in the study when referencing elementary school teachers. The article defined the proportion of 
"well-qualified teachers" as the proportion holding state certification and the equivalent of a major (either an 
undergraduate major or masters degree) in the field taught. For elementary teachers, the equivalent of a major was 
defined an elementary education degree for generalists who teach multiple subjects to the same group of students or 
as degree in the field taught for elementary specialists (e.g. reading, mathematics or mathematics education, special 
education). The study defined "out-of-field" for elementary teachers in the same way it was defined for secondary 
teachers: holding less than a minor or the equivalent in the fields described above (elementary education in the case of 
generalists or the specialist field (e.g. reading or mathematics in the case of specialists). 



27. For some mysterious reason, Walsh also tries to make a point that I differentiate (wrongly in her view) between 
cognitive ability or IQ and verbal ability (see her footnote 14, p. 8), despite the fact that this is a standard distinction 
in the literature made by many of the analysts Walsh herself quotes for support of the importance of verbal ability 
measures. Few measurement experts would argue that IQ, as it was defined and measured in the 1940s and '50s, 
represents the same construct as verbal ability, as Walsh seems to be invested in proving. 

28. Walsh makes a hash of the research cited here on the relationship between teacher test scores and measures of 
teacher effectiveness, striving to prove that studies which found largely insignificant positive and negative 
relationships between NTE scores and student achievement at least did not find significant negative relationships. 
Since there is little disagreement about the value of having teachers demonstrate their basic skills and subject matter 
knowledge through either coursework or testing, I do not review each of these older studies here. 

29. In her separately-published appendix, Walsh states that, "In 1999, Darling-Hammond summarized the main point 
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of this article as a call for using student achievement as the measure of teacher quality." In fact, in Darling-Hammond 
(1999), I cited this review for an entirely different point. I cited it for the proposition that "Teachers’ abilities to 
structure material, ask higher order questions, use student ideas, and probe student comments have also been found to 
be important variables in what students learn." 

30. Cited in the separately-published appendix entry 88, p.50. 
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Abstract 

The academic achievements of students taught by under-certified primary school teachers were 
compared to the academic achievements of students taught by regularly certified primary school 
teachers. This sample of under-certified teachers included three types of under-qualified 
personnel: emergency, temporary and provisionally certified teachers. One subset of these 
under-certified teachers was from the national program "Teach For America (TFA)." Recent 
college graduates are placed by TFA where other under-qualified under-certified teachers are 
often called upon to work, namely, low-income urban and rural school districts. Certified 
teachers in this study were from accredited universities and all met state requirements for 
receiving the regular initial certificate to teach. Recently hired under-certified and certified 
teachers (N=293) from five low-income school districts were matched on a number of variables, 
resulting in 109 pairs of teachers whose students all took the mandated state achievement test. 
Results indicate 1) that students of TFA teachers did not perform significantly different from 
students of other under-certified teachers, and 2) that students of certified teachers out-performed 
students of teachers who were under-certified. This was true on all three subtests of the SAT 9 — 
reading, mathematics and language arts. Effect sizes favoring the students of certified teachers 
were substantial. In reading, mathematics, and language, the students of certified teachers 
outperformed students of under-certified teachers, including the students of the TFA teachers, by 
about 2 months on a grade equivalent scale. Students of under-certified teachers make about 
20% less academic growth per year than do students of teachers with regular certification. 
Traditional programs of teacher preparation apparently result in positive effects on the academic 
achievement of low-income primary school children. Present policies allowing under-certified 
teachers, including those from the TFA program, to work with our most difficult to teach 
children appear harmful. Such policies increase differences in achievement between the 
performance of poor children, often immigrant and minority children, and those children who are 
more advantaged. 
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There has been growing interest in ’’teacher quality," a catch phrase for a host of teacher characteristics, including a 
teacher’s content knowledge, classroom behavior, academic ability, advanced degree work, salary, and teacher 
education experiences. Among the many characteristics under investigation as an indicator of teacher quality has 
been teacher certification. This study examines the effects of different kinds of teacher certification on student 
achievement. Reviews of this issue may be found in Darling-Hammond, 1999 and 2002; Evertson, Hawley & 
Zlotnik, 1985; and Ashton, Crocker, & Olejnik, 1986. 

In Arizona, a state with a strong commitment to standards based reform, policies were needed to ensure that quality 
teachers would be available for students to meet the new and more rigorous mandated standards. Thus the Arizona 
Educator Proficiency Assessment (AEPA) was developed as a tool in the state certification process to ensure the 
quality of new teachers. One part of the test purports to measure teachers’ professional knowledge, including 
pedagogy, teaching methods, and educational theory. The second part of the test covers content knowledge, either 
elementary content, or for secondary teachers, a subject specific content area. A passing score on the test, clearance 
by the police of any criminal record, and an accredited university’s recommendation that a person is prepared to work 
as a classroom teacher earns a regular certificate to teach from the State. 

Arizona’s efforts are part of a national movement to improve the quality of teachers through assessments like the 
AEPA (’’Quality Counts," 2000; Higher Education Reauthorization Act, 1998). But not every district in the state or 
nation can find regularly certified teachers, giving rise to other policies that appear to work against the goal of 
increased teacher quality. For example, in Arizona and elsewhere, attempts at improving the quality of the teaching 
force seem contradicted by the continuing practice of issuing emergency certification (see "Quality Counts," 2000; 
Olson, 2000). Critics of hiring uncertified teachers ask whether complex, standards — based reforms can be enacted 
with teachers who are, to varying degrees, untrained. Supporters of hiring uncertified teachers claim that the 
advantages of traditional teacher education programs are unproven, and some question, as well, whether such training 
is even necessary. Stated in its simplest form as a research problem the question is: "Do students taught by teachers 
with emergency certification learn as much or achieve as well as students who are taught by regularly certified 
teachers?" An answer to this simple question would inform us whether policies designed to improve teacher quality 
are being undermined by the simultaneous adoption of policies that allow the use of uncertified teachers. 

The dilemma associated with using uncertified teachers is not limited to Arizona where, currently, one out of six 
teachers are estimated to be uncertified (Go, 2002). For example, the Chicago "Sun Times" (Rossi & Grossman, 
2002) reports an audit by the Chicago Board of Education showing that 22 percent of teachers in the system’s 81 
probationary schools — those with the greatest academic needs and the lowest test scores — were not fully qualified to 
teach. These were teachers missing what the state calls "initial" or "standard" certificates. Other teachers were found 
with certificates, but they were teaching subjects they were not certified to teach. In all, 900 teachers, about one of 
every five in Chicago’s worst-performing public schools appeared unqualified to teach during the school year 2001- 
2002. New York State appears to be no different. Lankford, Loeb and Wycoff (2002) report that in a recent school 
year, in some New York schools, less than half the teachers held certification for the courses they taught. These 
schools were invariably urban and serving the poor. 

With the passage of the No Child Left Behind Act of 2001 (HRl), federal law will require schools to have a "highly 
qualified" teacher in every classroom by 2005-2006. Thus research on the effectiveness of uncertified and certified 
teachers takes on added significance as the designation of teachers as uncertified and certified becomes entwined with 
the evaluations of teachers so that the highly qualified can be distinguished from those less qualified. 



The review that follows explores the difference between licensure and certification, reasons for teacher certification, 
the role of certification in the professionalizaton of the teaching workforce, on-the-job training for teachers, and the 
assessment of beginning teachers’ competency. After those topics are discused briefly, research that bears on a broad 
range of certification issues and teacher effectiveness is discussed in somewhat more detail. These areas are all highly 
contested and interpretations of this literature are, more than usual, intertwined with the ideology of the researchers 
(Cochran-Smith & Fries, 2001). 

Licensure and Certification 

Teacher certification, at its core, is based on the need to ensure that every public school teacher has had rigorous 
screening and training and been judged qualified to teach. Certification is designed to protect the public from harm. 
But there is a difference between certification and licensure. Lawyers, cosmetologists, and physicians represent a few 
of the many professions that require a practitioner to hold a license in order practice their profession. The goal of 
licensing tests is to set a minimum level of competency. Professions that require licensure make it illegal for someone 
without a license to practice that occupation (Pybum, 1990). A person without a law or cosmetology license would be 
committing a crime if caught practicing law or working in a hair-dressing salon. No such legal protection is afforded 
the public when it comes to education. Teachers without certification are simply not allowed to use the title of 
"certified teacher" but there are no legal impediments for teaching without certification. This difference between 
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certification and licensure allows states to issue emergency certificates but not emergency licenses. Issuance of these 
emergency certificates produces considerable moral difficulty. A newspaper report on Chicago public schools makes 
this case dramatically (Rossi & Grossman, 2002). The journalists document that at Howland elementary school, one 
of Chicago's poorest: 

55 percent of teachers were "not fully certified" to teach all of their students.... That included four of 
six teachers in high-stakes classrooms, where kids must repeat a grade if they don’t do well on annual 
tests. All four held substitute certificates, although two were in teacher preparation programs. 

Assigning uncertified substitutes to low-scoring kids who face high-stakes tests "should be illegal," 
said Kati Haycock, head of the Education Trust, a Washington, D.C., research and advocacy group. 

"That’s educational malpractice." However, in Chicago, no policy governs who can teach such 
students. 

The "Philadelphia Inquirer" wondered what the city of Philadelphia was going to do about the same problem 
(Hundreds of teachers, 2002). President George W. Bush had just asked America to ensure that there was a highly 
qualified teacher in every classroom. But Philadelphia has 30,000 students in classrooms where teachers are 
uncertified and the city cannot figure out how to solve that problem. President Bush did not mention that while 
Governor of Texas, during the 1996-97 school year, he allowed 760,000 of the state's 3.8 million students to be taught 
by uncertified teachers. Nor did he note that the students with uncertified teachers were found not to do as well on the 
state achievement tests as did students in the classes of regularly certified teachers (Students of certified, 1999). 
President George W. Bush has now passed on to the nation the problem that Governor George W. Bush could not 
solve. 



Reasons for Certification 

Those who defend the process of teacher certification claim it is a necessary component in the development and 
maintenance of the profession of teaching, as well as the means by which the state can ensure the quality of those who 
enter the profession. Wise (1994a) notes, however, that there are two methods of controlling entry into the 
profession, professional control and popular control. 

Professional control allows the teaching profession to monitor who becomes a teacher. By specifying standards for 
certification and through various political mechanisms, the profession controls the quality of teachers who enter the 
profession. When professional control is present we often see teacher input in the design of teacher certification 
tests. On the other hand, popular control allows public demand to control who is placed in classrooms, with much 
less concern for their qualifications. Emergency certificates to teach during times of "shortages" are an example of 
popular control. Wise (1994b) advocates professional control as the primary means to allow the promotion of 
teaching within the economic sector. He believes that without certification teaching becomes a trade rather than a 
profession. 



Among other characteristics, a profession is also defined as possessing a distinct body of knowledge and having 
control of the education and licensing of its members (Pratte & Rury, 1991 ; Burbules & Densmore, 1991). Labaree 
(1992) describes professionalization as the ability to demonstrate formal knowledge and to have autonomy in the 
work place. He explains that any occupational group: 



...must establish that it has mastery of a formal body of knowledge that is not accessible to the 
layperson and that gives it special competence in carrying out a particular form of work. In return, 
the group asks for a monopoly over its area of work on the grounds that only those certifiably capable 
should be authorized to do such work and to define appropriate practice in the area (p. 1 25). 



Both Wise (1994b) and Roth (1994) fear that demand-based policies allowing for uncertified teachers can be 
devastating to the profession. They argue that such policies are likely to reduce the quality of teaching, lower the 
livable wage of teachers, and change the resources that are spent on and in schools. In effect, downgrading the 
importance of certiiication and training prevents teaching from m.eeting one of the criteria by which an occupational 
group calls itself a profession. 



[A] shift in locus of preparation [from university programs to alternative certification programs] 
moves teaching in the direction of trade. On-the-job training is not characteristic of a profession. 

Dismissing the requirement of professional preparation and a credential prior to practice is also 
uncharacteristic of a profession (Roth, 1994, p. 267). 

This battle over control of training is not new. For over 150 years who certifies teachers and how that certification is 
to be done has been a topic of intense debate. At all times, as might be expected, professional educators have fought 
to control the process, using medicine and law as their models (Angus, 2001). 



On-the-job Training and Teacher Certification 



To counter the argument that teachers can learn all that is required to be effective on the job, Darling-Hammond 
(2000), Howey and Zimpher (1994), and others argue that there is an inadequate amount of supervision and training 
provided to novice teachers by schools. Principals and veteran teachers who could serve as mentors generally do not 
have the required skills, training, or time to provide novices with quality supervision for on-the-job training. With 
few exceptions, school districts do not now have access to the additional resources needed for the training of teachers, 
and it is unlikely that such resources can be obtained. Hawley (1992), articulating the views of many others, claims 
also that there is a body of subject-matter content and subject-matter method, as well as skills and pedagogical 
knowledge, that needs to be learned prior to teaching. He and other teacher educators argue that it is unlikely that 
someone without training in subject matter methods could get in front of a class of students and be a successful 
teacher. This group of scholars rejects the idea that effective teaching can be learned on the job. 

Ordinarily, certification should assure the public that a minimal level of competency has been achieved, thereby 
insuring that unqualified people are not practicing the profession. Darling-Hammond (2000) believes the extant data 
supports that claim. For example, in an analysis of state level data she found the percent of new teachers in a state 
who were uncertified correlated negatively with performance on six different state assessments conducted by the 
National Assessment of Educational Progress. These correlations ran from -.40 to -.63. She found equally large 
positive correlations for the percent of teachers in a state having regular certification and a major in the field in which 
they were teaching, again using six NAEP data sets. These correlations ran between +.61 and +.80. Even for state 
level data the correlations Darling-Hammond found seem to be quite large, allowing her to assert with confidence that 
students achieve better when they have certified teachers as instructors. 

Certification and Competency Testing for Novice Teachers 

There are disputes about what should be covered on teacher certification exams because there is conflict about what is 
necessary for teachers to know in order for them to be effective. Nevertheless, and despite the problem of defining 
these areas unambiguously, teacher certification exams currently focus on measuring basic skills, content and 
pedagogical knowledge (Kearns, 1984). 

In 42 states, "Candidates [for certification] are required to pass one or more tests of basic skills, general knowledge, 
subject matter knowledge, or teaching knowledge" (Mitchell, Robinson, Plake, & Knowles, 2001, p. 44). But critics 
maintain that the basic skills certification tests "measure verbal and mathematical achievement at about the 10th grade 
level. And many states set their passing scores so low that virtually anyone can succeed" (Olson, 2000). Even 
Minnesota, usually one of the highest scoring states in the nation on standardized tests, has a teacher basic skills test 
that currently passes over 99% of the applicants (Scores needed, 2002). The Arizona basic skills assessment was also 
of this kind. It had only a 1% failure rate (Crehan, Hess, Lawrence, & Sabers, 2000). This basic skills test was 
abandoned, in part, because of its low failure rate and also because of adverse impact on some of Arizona's minority 
group candidates. The low failure rates nationwide suggest that teacher competence testing in basic skills areas is 
much more about symbolic politics (the need to appear as if standards have been put in place), and a lot less about 
genuine and systematic attempts to upgrade the quality of the profession. 

In the area of pedagogical skills and methods, the test developers often determine what good teaching looks like based 
on some definition of teacher quality. But teacher quality often is defined as having the necessary knowledge, skills, 
abilities and behaviors of a good teacher, a circular definition providing little guidance. Moreover, ideas about 
quality change from one decade to the next, as well as from one test developer to another, and the criteria for 
measuring teacher quality (the knowledge, skills, and abilities a teacher possesses) is not readily agreed on from 
person to person (Mitchell, Robinson, Plake, & Knowles, 2001). 

The National Research Council (Mitchell, Robinson, Plake, & Knowles, 2001) reviewed past and present definitions 
of teacher quality and competency, finding that past definitions of teacher quality emphasized teachers’ virtue. In its 
more modem form teachers are still expected to be role models for students, representing high standards of personal 
behavior and expected to transmit worthy cultural and education values. With no diminishment over time, it is still 
assumed that effective teachers possess certain personality traits, including enthusiasm, curiosity, and compassion, as 
well as democratic values. And, as always, effective teachers have been thought talented in teaching the prescribed 
curricula, an increasing concern after states have invested heavily in the development of standards and accountability 
systems. 

Today, most definitions of teacher competence from which assessments follow, are the product of three organizations, 
namely, the National Board for Professional Teacher Standards (NBPTS), the Interstate New Teacher Assessment 
Support Consortium (INTASC) and the National Council for Accreditation of Teacher Education (NCATE). The 
National Research Council (Mitchell, Robinson, Plake, & Knowles, 2001) notes that all three sets of standards of 
quality promoted by these organizations examine teaching in the light of student learning, a relatively new concern. 
While a focus on learning seems obvious to parents and politicians, this is a much more problematic an issue then it 
appears to be. The three organizations also acknowledge that teachers’ actions or performances depend on many 
different kinds of knowledge as well as the dispositions to use that knowledge. And they recognize that teachers must 
also work with others to support the learning and success of all students. The standards of competence described by 
the three organizations relate to a teacher's commitment to students and their students' learning, requiring that 




teachers: 



• should act on the belief that all students can learn; 

• should have deep subject matter knowledge about the substance and structure of their disciplines; 

• need to manage and monitor student learning, identify learning goals and choose from teaching styles to meet 



• need to be reflective about their teaching, evaluate their decisions and experiences to make adjustments in 
their teaching; 

• must be part of a larger community consisting of school staff, parents, and the broader non-parent 



As the National Research Council notes, the standards currently in use "...illustrate the wide range of knowledge, 
skills, abilities and dispositions that contemporary educators believe competent teachers must possess and 
demonstrate in the classroom" (Mitchell, Robinson, Plake, & Knowles, 2001 , p. 3 1). Given the wide range and 
lengthy descriptions of the knowledge, skills, abilities and dispositions that the various assessors of teachers’ 
competence have considered necessary to begin to teach, it seems anomalous that arguments to drop teacher education 
or to provide emergency certificates would have any credibility at all. Test developers find effective classroom 
teaching to be extremely complex, requiring the coordination of many different kinds of skills and dispositions, many 
of which cannot be observed directly. It is hard to imagine that an occupation with these characteristics can be 
effectively learned on-the-job. 

This very same occupational complexity also handicaps the developers of teacher certification testing, leading Sabers 
to note that, "test developers and researchers haven’t done a good job of telling the public that they can’t measure 
everything with a test" (in Crehan, et al., 2000). The public believes that a certification exam will eliminate poor 
teachers from schools and in essence guarantee that teachers who pass these exams are of high quality. But we do not 
yet have such tests. At present, it is fair to say that many aspects of teaching cannot be assessed by using a multiple 
choice or essay exam, and if performance tests of teaching were used such testing would be prohibitively expensive 
(Crehan, et al, 2000). 

Validity problems w^ith certification tests 

. All teacher certification exams have problems with construct, content, consequential and criterion-referenced 
validity (Laczko-Kerr, 2002). In brief, there is no evidence that the construct measured in teacher certification exams 
is understood, in addition, course content varies more widely across teacher training institutes then, say, law schools 
or medical schools. This invites criticisms about the content validity for teacher certification exams. Additionally, 
certification tests limit the pool of potential teachers based on race and ethnic background (Mumane, 1991). As the 
need for teachers of color increases because of the increase in children of color in our public schools, the numbers of 
minority teachers seems to be decreasing (Gitomer et al., 1999). One reason for this is the increased requirements for 
insuring teacher quality, including certification testing for teachers. But these exams have an adverse impact on the 
teacher supply and this raises concerns about the consequential validity of the exams. Finally, teacher certification 
exams do not appear to have criterion-related or predictive validity (Smith & Hambleton, 1990; see also Glass, 2002). 
Certification tests simply do not predict success in teaching. Rather, their intent is to screen out certain applicants 
from the teaching pool (Sabers in Crehan, et al., 2000). Jaeger, quoted in the National Academy of Science/National 
Research Council report has an additional concern, namely, that ’’ the sorts of experimental or statistical controls 
necessary to produce trustworthy criterion-related evidence [are] virtually impossible to obtain" (Mitchell, Robinson, 
Plake, & Knowles, 2001 , p. 72). These problems with validity, particularly predictive validity, seem to bolster the 
arguments of those who support emergency certification. If the tests cannot predict teaching competency, they argue, 
why must they be required for certification? The answer offered to that question by supporters of teacher certification 
tests is that passing the tests ensures familiarity with a broad teacher education curriculum, without which beginning 
teachers would not do well This debate can be restated as a version of the simple question we noted above: Does 
teacher education and the certification that accompanies such programs make a difference in the achievement of 
students? 

Research on Certified Teachers and Student Achievement 

Three major areas of research are salient for understanding the importance of certification. First is the research on the 
effects of certification regarding teachers’ content knowledge, particularly mathematics and science knowledge, as it 
affects student achievement. A second area of research deals with the effects of certification regarding a teachers’ 
pedagogical knowledge, and its effects on student achievement. It is clear that the federal government is having 
troubles deciding on the relative importance of these two areas, paying lip service to the latter but more often 
endorsing the former. As "Washington Post" reporter Jay Mathews notes (2002), first the Bush administration pushes 
through an education bill that demands a "highly qualified" teacher in every classroom. Then the administration 
releases a report arguing that the nation’s education schools spend too much time on classroom methodology. 



these goals; 



community. 




Mathews points out that mixed messages are being sent to the public. But in fact, they aren’t very mixed. 



Education Secretary Roderick Paige and other Education Department officials claim that schools of education need to 
spend less time on pedagogical issues and spend more time worrying about whether teachers understand what they 
teach. The current mantra of federal educational administrators seems to be "You can’t teach what you don’t 
know" (Mathews, 2002). Not mentioned explicitly, but implied, is that schools of education should have little role in 
the training of teachers. Secretary Paige’s comments are all the more puzzling from some one who advocates 
evidence- based research. His own Department of Education recently requested a review of "rigorous empirical 
research’’ on teacher preparation (Wilson, Floden & Ferrini-Mundy, 2002). The authors of this government 
commissioned report concluded that subject matter knowledge is not sufficient for effective teaching to take place. 

[The studies reviewed] suggest that the subject matter preparation. ...prospective teachers currently 
receive is inadequate for teaching toward high subject matter standards, by anyone’s definition. 

[Without training in pedagogy] it appears that prospective teachers may have mastered basic skills 
but lack the deeper conceptual understanding necessary when responding to student questions and 
extending lessons beyond the basics (p. 192). 

The third area of research focuses on two sub-areas that both deal with traditional certification and alternatives to it. 
One of these areas of research is on the effects of regularly certified teachers teaching in or out of their area of 
expertise. In this literature in-field vs. out-of-field teaching performance is compared, such as when an English 
teacher is assigned to teach algebra. Out-of-field teaching can be viewed as teaching without the appropriate 
certification to do so. The second sub-area is concerned with the effects of alternatively certified teachers in 
comparison to traditionally certified teachers. Present government policy has decided that alternate means to 
certification are appropriate, with officials claiming that: 

[T]here is no evidence that lengthy preparation programs achieve [their] goals any better than 
streamlined programs that quickly get talented teachers into the classroom.. ..Requiring excessive 
numbers of pedagogy or education theory courses acts as an unnecessary barrier for those wishing to 
pursue a teaching career (Mathews, 2002). 

Our evaluation of this literature, reviewed in more detail below, is that there is sufficient evidence to conclude that 1) 
subject-matter knowledge is an important, but not sufficient, factor in a certified teacher's success with mathematics 
and science students in the upper grades; 2) that teachers who have training in pedagogy outperform teachers without 
such training; and 3) that traditionally certified teachers teaching in their area of certification outperform both 
certified teachers teaching out-of-field and alternatively certified teachers. The data on these issues, however, is 
certainly not unequivocal, and dissenters to all these conclusions exist (see Ballou & Podgursky, 1999; Peck, 1989; 
Miller, McKenna, & McKenna, 1998). We look at these literatures next. 

Teacher Subject-matter Knowledge 

. Studies related to teacher subject matter often either evaluate 1) whether a major or minor in a subject area, e.g., 
mathematics, effects student achievement (Hawk, Coble, & Swanson, 1985; Goldhaber & Brewer, 1996; Monk & 
King, 1994); 2) whether a passing score on a certification exam provides evidence that certain subject matter has been 
mastered or that certification affects student achievement (Ashton & Crocker, 1987; Byrne, 1983; Strauss & Sawyer, 
1986; Glass, 2002); or 3) whether advanced degrees, e.g., master’s degree, or professional development increase 
student achievement (Goldhaber & Brewer, 1996, 2000; Fetler, 1999; Ehrenberg & Brewer, 1994). Each of these 
areas of evidence will be reviewed separately. 

Researchers have usually, though not always shown that having a major or minor in mathematics or science is 
beneficial to student achievement in those content areas. Hawk, Coble, and Swanson (1985) provide research on that 
issue obliquely, by comparing in-field and out-of-field teaching; concluding that student achievement, for general 
mathematics as well as algebra, is greater for students who are taught by teachers certified in mathematics (in-field 
teachers, possessing a major or minor) than is the achievement of students taught by teachers certified in some other 
content area (out-of-field teachers, neither a major or minor in the area). The researchers hypothesize that the greater 
success of these in-field teachers’ appears to be their greater ability to successfully impart content specific knowledge 
to students, as compared to their out-of-field counterparts. It is important to note that these studies compare teachers 
who hold a standard teaching certificate in their subject area (indicating specialized content knowledge training) with 
teachers who also hold a standard teaching certificate in another subject area (indicating a lack of specialized content 
knowledge training). The study supports the case for certification in a content area, and suggests deficiencies can be 
expected among those who are teaching in areas for which they are not prepared. 

From their research Goldhaber and Brewer (1996) conclude that "in mathematics and science, teacher subject-specific 
training has a significant impact on student test scores in those subjects" (p. 206). These same researchers go on to 
say that their results suggest that it is subject-specific training, not teacher ability that leads to such findings. These 
authors believe that achievement in technical subjects can be improved by a cessation of out-of- field teaching. The 
generalizability of these results to the humanities and for teachers in the primary grades is unknown. 
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Monk and King (1994) also evaluated subject-matter preparation and student performance. In an earlier analysis 
Monk (1994) had found that there was a "positive relationship between the number of subject-related courses in a 
teacher's background and subsequent performance gains of these teachers students within the indicated subject 
area" (as cited in Monk & King, 1994, p. 36). Continued investigation of this phenomena revealed interactions, 
among them, that, "low-pretest students' performance gains in mathematics were more sensitive to the mean level of 
their previous teachers preparation than were the high-pretest students" (p. 56). This suggests that lower achieving 
students will profit more from teachers who are well prepared in their subject matter, than might better achieving 
students. Thus policies that promote uncertified teachers as the instructors of the poorest and the lowest achieving 
students, which is the way those policies are usually realized, may be particularly harmful. 

Byrne (1983) provides a review of thirty studies that relate teachers' subject matter knowledge, measured by subject 
knowledge exams or coursework taken, to student achievement. These results were contradictory. A majority of the 
studies showed a positive relationship (17), while a large number (14) showed that no relationship existed. Byrne 
does not provide more than a tally analysis of the studies included, which is insufficient given the capabilities of 
meta-analytic research. A re-analysis using meta-analysis would be helpful. 



The National Teacher Exam (NTE) was once used as a measure of subject matter knowledge and was extensively 
studied. It measured both subject matter content knowledge and pedagogical knowledge. Quirk, Witte, and 
Weinberg (1973) found only a single study, Lins (1946), in which NTE scores were correlated positively with 
students' average gain in performance from pretest to posttest. But this finding must be reconsidered in light of the 
work of Strauss and Sawyer (1986). They analyzed the determinants of average student performance on standardized 
exams and found that a " 1 % increase in teacher quality, ceteris paribus, as measured by standardized test scores 
[NTE], is accompanied by a 3-5 % decline in the level of failure or rate of failure of students on high school 
competency examinations" (p. 41). Simply put, increased scores by teachers on the NTE exams, reflecting increased 
subject matter knowledge and increased pedagogical knowledge, decreased student rates of failure. Research on the 
PRAXIS tests, successor to the NTE, has been conducted by Gitomer, Latham, and Ziomek (1999), and will be 
reported below. 

Teachers' advanced degrees are another indicator of subject matter competency. Goldhaber and Brewer (1996), as 
part of the study reported above, also found that teachers' degree level is significantly related to student achievement 
in the area in which the degree was obtained. However, when a general production function model is used, teachers 
with master's degrees appeared to be no more effective than teachers without advanced degrees. Results varied 
depending on the statistical models that were used to analyze teacher effects. Goldhaber and Brewer (2000) report 
that mathematics students who have teachers with bachelors or masters degrees in mathematics have higher test 
scores than students of teachers without these degrees. They report, however, that there is no comparable impact of 
degree in science. Fetler (1999) confirms the findings in mathematics, noting that "Schools with more experienced 
and more highly educated mathematics teachers tend to have higher achieving students" (p. 1 3). But of course, 
higher achieving students have access to better schools, and thus these kinds of studies require caution when 
interpreting them. 



Ehrenberg and Brewer (1994), however, report unambiguously that teachers' degree level does matter. "The greater 
the percentage of teachers with at least a masters degree... the higher black students' scores are [on measures of 
mathematics, reading, and vocabulary that are associated with the High School and Beyond study]" (p. 10). On the 
other hand, with a Texas sample, Rivkin, Hanushek, and Kain (2000), note just the opposite, namely, that there "is no 
evidence that a masters degree raises teacher effectiveness" (p. 3). However, these researchers also report "teacher 
quality is a very important determinant of the quality of education" (p. 30). Thus the conclusions in this report appear 
contradictory. 

Kain (1998) also studied this issue. He reports that in his earlier research, teachers accounted for at least 7% of total 
variation in student achievement, indicating that a set of teacher characteristics, including certification and training, 
affect student achievement. 



To analyze the affect of teacher degree on student achievement, Goldhaber and Brewer (2000) used data from the 
National Educational Longitudinal Study of 1998 (NELS: 88), which provided information about students and their 
teachers in 10th and 1 2th grades. Students were surveyed as well as tested on one or more mathematics, science. 
English/writing and history exams. The researchers found that "having a degree in education has no impact on 
student science test scores and, in mathematics, having a BA in education actually has a statistically significant (at the 
10% level) negative impact on mathematics scores of students" (pp. 1 38-139). Goldhaber and Brewer's research 
focused on secondary grades. The complexity of the content taught in secondary mathematics classes is undoubtedly 
greater than that taught in elementary school, so advanced training in mathematics may be required to effectively 
transmit that content. That same depth of subject matter knowledge may not be required for elementary teaching. 

There is some support for this hypothesis, although research on the effects of advanced degrees and subject matter 
majors for primary grade students is scanty. One such study, however, was published by the National Center for 
Educational Statistics (NCES) (as cited in Hawkins, Stancavage, & Dorsey, 1 998). These results concerned fourth- 
grade mathematics students who took the National Assessment of Educational Progress (NAEP). The researchers 
note that "fourth-grade students who were taught by teachers with an undergraduate or graduate minor in mathematics 
or mathematics education did not perform better on the 1996 mathematics assessment than students whose teachers 
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had an undergraduate or graduate major in education” (p. 12). But the NCES report also states that, unlike fourth- 
grade, eighth-grade students who were taught by a teacher with a major in mathematics outperformed students taught 
by teachers with majors in education. 

One conclusion to be drawn from these data is that a teacher's depth of knowledge in a subject matter influences 
students' achievement more in the upper grades than the primary grades. But the broader conclusion might be that as 
the content in a subject matter area becomes more complex, teachers need a much deeper knowledge of that subject 
matter area to help their students learn at high levels. It is not grade level per se, we think, but the complexity of the 
ideas to be taught that requires the specialized subject matter knowledge of a teacher. Thus the claim made by TFA, 
that an individual with a subject matter major from an elite college who elects to teach elementary school without 
certification is bringing great strength to the schools, may be questioned. It seems probable that after basic college 
level content is mastered, after some threshold of competency in a subject matter domain is crossed, as it is for most 
college majors, then pedagogical training for teaching in the elementary grades is more important to success than is 
content knowledge. Support for this interpretation comes from Rowan, Correnti and Miller (2002), in their study of 
teacher subject matter competency in the early grades. We look at this issue next. 

Professional Knowledge/Pedagogical Content Knowledge 

The findings from research that examines a teachers' level of education related coursework and their effectiveness 
with students is extensive, but often contradictory. We believe, however, that some interpretations of this literature 
are possible, though we note that better research in this area is possible and badly needed, 

Ferguson and Womack (1993) found that the amount of education coursework teachers completed explained about 
16% of the variance in teaching performance, as measured by supervisor evaluations; this was more variance 
accounted for than with teachers' content knowledge, as measured by NTE specialty scores. This research suggests 
that education coursework is a strong predictor of teaching effectiveness, over and above grade point average in a 
teachers' major and their NTE specialty scores. In their review of research on this same issue Ashton, Crocker, and 
Olejnik (1986) also found education coursework to have a significant relationship to teacher performance. 

More recent research by Wenglinsky (2002), on the link between teacher quality and student performance, supports 
the belief that teacher inputs do influence student performance. He notes that the greatest influence on student's 
achievement comes from classroom practices and the professional development that supports them, Wenglinsky’s 
research indicates that "regardless of the level of preparation students bring into the classroom, decisions that teachers 
make about classroom practices can either greatly facilitate student learning or serve as an obstacle to it" (p, 7), That 
is, teacher pedagogical decisions and activities (which are separate from but not unrelated to teacher subject matter 
knowledge) independently make a difference in student achievement. 

Rowan and colleagues (Rowan, Correnti & Miller, in press) reached similar conclusions. These researchers found 
relatively large effects on young students that could be attributed to teachers, independent of school, social class, 
previous achievement, and so forth. For any given year, looking at a single score, at a single point in time, teachers 
accounted for 4% to 1 8% of the variance in student's reading and mathematics achievement. This yielded effect sizes 
of .21 to .42. Across years, looking at student growth, the effects of teachers on students were magnified. Analysis of 
the teachers' effects on student growth in reading yielded effect sizes of from .77 to .88. The teachers' effects on 
growth in mathematics achievement were equally impressive, yielding effect sizes of between .72 and .85, The effect 
of teachers' characteristics on student achievement growth, across time, is roughly three times greater than they are on 
student achievement status measured at only one point in time. When searching for which teacher characteristics 
make a difference, these investigators found that the most consistent predictor of young children's achievement was 
teacher experience. Experience was found to be a much better predictor of student achievement than was subject 
matter competency. Here again we see the relative importance of pedagogy over content knowledge in influencing 
the achievement of young children. 



What is often not discussed in research reports connecting some teacher quality variable and student achievement is 
that the great bulk of a teacher's pedagogical training and understanding of beneficial classroom practices is provided 
in their teacher training programs. Clearly experience matters; but that means that preparation to profit from 
experience must matter as well. And that suggests that the experience gained from intensive student teaching, over a 
sufficient time period, might also matter. Such experiences are provided as a matter of course in most traditional 
teacher certification programs, and are missing from most alternative and emergency certification programs. Without 
adequate teacher training, then, emergency certified teachers and other under-certified teachers could retard student 
learning as they engage in teacher learning. We examine teacher experience in more detail next. 



Teacher Experience 



Teacher experience is another teacher quality variable that influences student learning and is indirectly related to 
issues of certification. Hawkins, Stancavage, and Dorsey (1998) report that in the 1996 AEP analysis, students who 
were taught by teachers with less than 5 years of teaching experience performed below the level of those students 
whose teachers had 6-10 years or 25 or more years of experience (p. 22). Fetler (1999) also supports the finding that 
number of years teaching is positively related to student test scores. Lopez (1995), using a large data set from Texas, 
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reports that teachers require about 7 years of experience in order to be able to maximize their students' test 
performance. Similarly, Rivkin, Hanushek, and Kain (2000) report that there are small but significant relationships 
between student achievement gains and teacher experience: "The teacher experience investigation concentrates on 
entering teachers and supports the notion that those in the first two years of experience do worse than more 
experienced teachers. New teachers' average student gains are lower by roughly 20 percent of a standard deviation in 
both 4th and 5th grades" (p. 27). They also report that 4th and 5th grade student achievement on the Texas 
Assessment of Academic Skills (TAAS) is effected by overall teacher experience. The results indicate a small but 
significant relationship between achievement gains and teacher experience. The study of Rowan et al., (in press) cited 
above, supports this conclusion as well. 

The point of this discussion about experience is that virtually all university teacher certification programs include both 
early field experiences and student teaching in their curriculum to provide experience to novice teachers. While we 
have little empirical evidence to determine what kinds and amounts of experience are the most beneficial, it seems 
likely that teacher induction programs that offer little or no experience will be deficient. This is a criticism of the 
TFA program and any other programs supporting emergency or alternative certification that allow un-experienced and 
inexperienced teachers to become classroom instructors. Reviewing similar literature for the Department of 
Education, Wilson, Floden and Ferrini-Mundy (2002) conclude that the parts of the teacher education experience that 
make a difference are unknown, but that "the research suggests that there is value added by teacher preparation (p. 
194)." They also state that clinical experiences and field-work, such as that provided through student teaching, are 
seen as powerful forces — maybe the most powerful force — in programs of teacher preparation. 

Interestingly, if a state policy provides for emergency certification to teach for only a short period of time, they may 
do a disservice to students, since it is through experience that teachers acquire their competency. The logic is this: It 
may be wrong to employ emergency certified teachers, but to dismiss them solely on the basis that they served two 
years, the maximum for an emergency certificate in some states, is to negate and reject how much they may have 
learned in that time. On a case-by-case basis, it may be better to decide if an emergency teacher has been reflective 
about his or her experience and thereby learned enough to be effective. It may compound the original error to dismiss 
them after a short period of time. 

The review of research on content knowledge, pedagogical knowledge, and experience, given above, focuses on 
where these abilities and characteristics of teachers are to be learned, and in what mix, but there is no major dissent 
about their importance for student learning. Wenglinsky (2002) makes this case best using data from the National 
Assessment of Educational Progress (NAEP) to examine the role teachers and their instructional practices play in 
influencing student achievement. He summarizes his findings as follows: "The study finds that the effects of 
classroom practices, when added to those of other teacher characteristics, are comparable in size to those of student 
background, suggesting that teachers can contribute as much to student learning as the students themselves." 

Appropriate Assignment of Certified Teachers 

Some research on certification status supports the fact that teachers who are certified and teaching in the area in 
which they are certified outperform teachers who are less than fully certified and teachers who have no certification 
(Darling-Hammond, Berry, &Thoreson, 2001; Darling-Hammond, 1992; Fenstermacher, 1992; Evertson, 1984). 
Unfortunately, however, not all certified teachers are assigned to teach in the areas for which they have been trained 
(Goldhaber & Brewer, 1996; 2000; Ehrenberg & Brewer, 1994). In fact, large numbers of the certified teaching staff 
are assigned duties for which their certification is irrelevant. This kind of out-of-field teaching is exemplified in the 
elementary grades when a fifth grade teacher is assigned to teach a second grade class, or in high school, when an 
English teacher is assigned to teach an American History or a biology class. The most cited reason for out-of-field 
teaching is a lack of fit between the teachers on the staff and the teaching assignments that an administrator must 
make (Ingersoll, 1999a). 

In some curricula areas such as bilingual and special education, science, and mathematics, there is a teacher shortage. 
This appears to be caused by increased student enrollments, retirements due to the "graying" of the teaching 
profession, increased turnover by teachers due to increased difficulties in carrying out their jobs, and the increased 
remuneration for mathematicians and scientists for work in other fields. If faced with difficulty filling positions, 
school boards and administrators think of three solutions: they hire less qualified teachers, they assign teachers trained 
in another subject area to teach in the understaffed areas, or they make extensive use of substitute teachers. 

There are many problems with the teacher shortage explanation for out-of-field teaching. First of all, this explanation 
does not provide an answer for why large numbers of out-of-field teachers exist in fields like English and social 
studies, two areas of teaching that have long had a surplus of teachers. Another problem with this explanation is that 
it has only been within the past few years that schools have had trouble filling teaching openings, while the problem 
of out-of-field teaching has been with us for a significantly longer period of time. Finally, the teacher shortage is 
based on the assumption that there are too few teacher candidates. But in fact, the biggest difficulty is that teachers 
are choosing to leave their profession for other jobs (Ingersoll, 1999a). Ingersoll (1999a) comments, "Out-of-field 
teaching is common, I believe, because it is not only legal but also more convenient, less expensive and less time 
consuming than the alternatives" (p. 7). 

Ingersoll (1996) evaluated data from the School and Staffing Survey to determine the proportion of teachers who 




teach out-of-field. He provides data indicating that one-fifth of public school students enrolled in 7th- 12th grade 
English courses are taught by teachers who did not have at least a minor in English or some other closely related 
field. Of the students enrolled in 7th- 12th grade mathematics courses, more than a quarter are taught by teachers 
without a minor in mathematics or mathematics education. The results are less drastic in other areas of course work. 
In Arizona, 35% of teachers in English, 39% of teachers in Math, 35% of teachers in Social Studies and 27% of 
teachers in Science were assigned to teach secondary courses without a major in that subject area ("Quality Counts," 



Research also supports the belief that out-of-field teaching is related to levels of school poverty (Ingersoll, 1996; 
1999b; Haycock, 2001). Ingersoll (1996) reports, ’’ in no fields did high-poverty schools have less out-of- field 
teaching than did low poverty schools, while in several fields, students in high poverty schools received distinctively 
more out-of-field teaching than in low poverty schools" (p. 5). This trend is similar for students who are placed 
within different educational tracks in their courses. High track students are exposed to less out-of-field teaching than 
low track students (Goldhaber & Brewer, 2000; Rivkin, et al, 2000, Ingersoll, 1996), while "minority and poor 
students are disproportionately placed in lower track and lower achievement courses, [which] critics claim are taught 
by the least qualified" (Ingersoll, 1996, p. 1). Darling-Hammond (1997b) reports that in the most heavily minority 
schools and inner cities less than 50% of the teachers in mathematics and science are licensed and have a degree in the 
subject they teach. Darling-Hammond remonstrates that throughout the country we have the least qualified teachers 
teaching the most disadvantaged students, while the most qualified teachers are teaching the most advantaged 
students. 

At the secondary level the relationship between in-field teaching and student achievement is stated forcefully by 
Hawk, Coble, and Swanson (1985). TTiey conclude that: 

In field certified math teachers know more math and show evidence of using more effective teaching 
practices than their out-of-field counterparts. Further, and most important, students of in field 
certified math teachers achieve at a higher level than do students taught by out-of-field teachers (p. 



In short, a certified teacher teaching in the field for which they were prepared performs better than when assigned to 
areas for which they were not prepared. Preparation matters. 

Alternative Routes to Teaching 

Much of the research that draws attention to alternative certification programs does not adequately address the issue 
that many such programs are similar in both the level and rigor of training provided by traditional certification 
programs (Buck, Polloway, & Robb, 1995; Miller, McKenna, & McKenna, 1998; McKibbin, 1988; Bliss, 1992; 
Stoddard, 1992; Darling-Hammond, Berry, & Thoreson, 2001). On the other hand, many of the alternative teacher 
training programs are poorly designed and administered, providing little in the way of appropriate training (Wilson, 
Floden, Ferrini-Mundy, 2002). Although increasing dramatically in number, there are currently no standards for 
assessing alternative certification programs. The large variability in alternative certification programs makes research 
on this phenomenon difficult. (Of course, to be equally fair, we must note the variability in traditional programs of 
teacher education, whose design and administration have also been noted by many to be equally slipshod. Even 
accredited programs have, in our opinion, some embarrassing design characteristics.) 

Advocates of alternative certification, however, claim that they provide teachers for urban and rural schools and in 
specific shortage areas, i.e., mathematics and science. Zumwalt (1991) summarizes research on several alternative 
certification programs and reports that they do attract teachers who are more willing to work in rural or urban poor 
districts than traditionally trained teachers. McKibbin and Ray (1994) also report that alternative certification 
programs attract people with subject matter majors like mathematics and science who are interested in teaching, but 
not interested in traditional teacher certification. 

Alternative certification is also seen as a cost-effective way to train people who did not or will not enroll in 
conventional undergraduate or graduate education programs. Such programs are cheaper (Zumwalt, 1991), as might 
be expected from programs that are shorter in duration and provide less instruction, supervision and assessment of 
their students. 

Proponents of alternative certification claim that these programs attract better quality candidates who are more 
academically able than those who attend traditional certification programs (Kanstroom & Finn, 1999). Participants of 
these programs are generally people who have majored in traditional academic subjects rather than education. It is a 
major assumption of alternative programs that subject matter content knowledge is more important to teaching than is 
education related coursework (Jelmberg, 1996). But some studies show that the teachers in alternative routes to 
certification have high drop-out rates from both the programs of instruction and from actual teaching. (Wilson, Floden 
& Ferrini-Mundy, 2002). Other studies show that alternative certification recruits in mathematics and science have 
lower grade point averages than recruits in traditional teacher education programs (Stoddart, 1992). Moreover, to 
date, these alternatively certified teachers have not demonstrated strong skills in their content area. Furthermore, 
teachers from alternative routes to education, including TFA teachers, when compared to those trained in more 
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traditional teacher education programs, report many more problems with their preparation programs. For example, on 
39 of 40 different questions the TFA teachers rated their preparation more poorly than did those who were trained in 
more traditional programs. The self-confidence and sense of efficacy of those prepared in traditional programs was 
higher than for those who came to teaching through alternative programs (Darling-Hammond, Chung, & Frelow, 
2002 ). 



When these facts about alternative routes to teaching are added to research that debunks the belief that subject matter 
knowledge is more important than education related coursework (Darling-Hammond, 2000; Gitomer, Latham, and 
Ziomek, 1 999; Monk and King, 1 994; Wilson, Floden & Ferrini-Mundy, 2002), we see evidence of an educational 
policy that must be seriously questioned. 



Proponents of alternative certification also make the claim that traditional certification programs attract mostly 
twenty-something, white women, a problem because the students of the public schools in urban areas are not often 
white, and some commonality and similarity in life experience is probably a better background for teaching positions 
in these more difficult settings. Proponents of alternative certification correctly note that they often attract a more 
diverse group of candidates, specifically men, older adults, minorities and retried military personnel (Bradshaw & 
Hawk, 1996; Eifler & Potthoff, 1998; Hawley, 1992; Houston, Marshall, & McDavid, 1993; Keltner, 1994; Kennedy, 
1991; Kwiatkowski, 1999); Natriello & Zumwalt, 1993; MacDonald, Manning, & Gable, 1994; and Stoddart, 1993), 
Wendy Kopp (1994), Teach for America founder, notes that short alternative certification programs allow young 
adults who are unsure of their career options an opportunity to serve students. 



Alternative certification may actually be an improvement over simple emergency certification, which allows almost 
anyone with a bachelors degree to teach without any preparation to speak of. But some researchers (Bradshaw & 
Hawk, 1996; Berry, 2001) are critical of the level of professional knowledge demonstrated by alternatively certified 
teachers. Alternatively certified teachers tend to have a limited view of curriculum and a lack of understanding of 
student ability as well as motivation; they experience difficulty translating content knowledge into meaningful 
information for students to understand; they are less effective planners of instruction; and they tend not to learn about 
teaching through their experiences. Research is also critical of the supervision and mentoring support that is given to 
preservice teachers in most alternative certification programs (Smith, 1991; Darling-Hammond, 1992; McKibbin & 
Ray, 1994; Bradshaw & Hawk, 1996; Jelmberg, 1996; Miles-Nixon & Holloway, 1997; Berry, 2001), It appears 
likely, from the extant research and criticism, that poorly run and short alternative certification programs, particularly 
those that do not provide much classroom experience and supervision, may actually not be any better than simply 
hiring emergency certified teachers with no teacher education experiences. 

Teach for America 

The most familiar of the alternative certification programs is Teach for America (TFA), This ambitious program 
recruits graduates from top universities, provides them minimal training, and places them in public school classrooms 
across the nation to teach. The public schools, however, are all in either rural or poor urban districts (Darling- 
Hammond, 1994). Research conducted on TFA has been less than encouraging. 



Four separate evaluations found that TFA's training program did not prepare candidates to succeed 
with students, despite the noticeable intelligence and enthusiasm of many of the recruits. Most 
criticism of a corps member's teaching behavior (classroom management was the greatest area of 
concern, followed by insufficient knowledge of the fundamentals of teaching and learning) was 
qualified by the cooperating teachers' perceptions of limitations of the program in providing the corps 
member with adequate practice or theory to be successful (Darling-Hammond, 1997a, p, 310), 



From an interview study by Stevens and Dial (1993), TFA teachers apparently decide to teach because they like 
working with children; they didn't have other options; and they felt that TFA was their best alternative given their 
"circumstances and indecisiveness at the time" (p, 70). 

Jonathan Schorr (1993), a former TFA teacher, describes the inadequate training and preparation that he and other 
TFA teachers received prior to being placed into schools. He notes, "just eight weeks of training ,,, is not enough for 
teachers" (p. 3 1 6). Schorr admits, "I was not a successful teacher, and the loss to the students was real and large" (p, 
318). Schorr offers the first-hand experience that makes Darling-Hammond (1994; 1997a; 2001) quite critical of 
TFA, specifically due to the program’s limited training of candidates, lack of evaluation, and the fact that such a 
program perpetuates the placement of poorly trained teachers with the most needy students in the nation. 



Raymond, Fletcher, and Luque (2001) conducting research for the Center for Research in Education Outcomes 
(CREDO), released a report evaluating the Teach for American program in Texas. The report compares scores on the 
Texas Assessment of Academic Skills (TAAS) of students taught by TFA teachers and non-TFA teachers, and lauded 
the performance of TFA teachers. However, the National Commission on Teaching and America's Future released a 
response outlining serious concerns with the research (see Darling-Hammond, 2001 and 2002). The most important 
of the criticisms is that the performance of the TFA teachers was. never compared to the performance of regularly 
certified teachers. The comparison used to assess the TFA teachers was other uncertified teachers, some of whom 
didn't even have four-year college degrees. 
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It should also be noted that when we tried to access the data for this report, we were informed from both the primary 
researcher and the Texas school district responsible for the data that it was not available for independent review. We 
were told that the data was not the property of the researchers who reported the study, nor did it belong to the district, 
and that neither had a complete data set to provide for independent analysis. In our opinion, therefore, it is 
appropriate to regard this report as irrelevant, given that the comparison used to assess TFA teachers was faulty, the 
data are not available for verification or replication by other scientists, and the report has not been published in a peer- 
reviewed journal. 



Traditional Teacher Certification 



In this section we discuss traditional programs of teacher certification, with the understanding that they vary 
enormously, as do the alternative certification programs. The Holmes Group (1986), active in teacher education 
reforms, reports that competent teaching consists of three elements: subject matter knowledge, systematic knowledge 
of teaching, and reflection on experience. Members of the Holmes group defended teacher education in the 
universities by arguing that they do, in fact, prepare people to successfully integrate these three elements into their 
professional lives. In their defense we note that teachers entering the field from university teacher education 
programs are generally more academically able than the average college student, while unlicensed entrants into 
teaching have significantly lower levels of academic achievement than most college students and were lower also then 
those who were prepared by the university to teach (Gitomer, Latham, & Ziomek, 1999). In fact, "reviews of 
research over the past thirty years, summarizing hundreds of studies, have concluded that even with the shortcomings 
of current teacher education and licensing, fully prepared and certified teaches are better rated and more successful 
with students than teachers without this preparation" (Darling-Hammond, 1997a, p. 308; Evertson 1984). Apparently 
disregarding this research. Former Secretary of Education Chester Finn proposes that the common sense route to 
boosting teacher quality is to simplify entry and hiring. Fundamentally, he argues, we should let anyone who wants to 
teach do so, and simply deregulate the teacher certification process (Kanstroom & Finn, 1999; Finn & Kanstroom, 
2000). Finn is also the president of the foundation that helped support the unverifiable Raymond, Fletcher, and Luque 
(2001) study that is so supportive of uncertified teachers. Nationally syndicated conservative columnist Thomas 
Sowell supports Finn's position. Sowell (2002) says bluntly that college of education courses are "drivel" and falsely 
reports that the academic record of those who enter teaching through the university route is deficient in comparison to 
almost all other college majors. 



The research martialed in support of prepared and certified teachers includes research demonstrating that teachers 
hired without preparation or only partial training experience difficulty meeting the needs of the students in their 
classrooms. Such individuals have more difficulties than fully prepared teachers do in accomplishing their day-to-day 
job requirements (Darling-Hammond, 1997a). Prospective teachers apparently perform better if they have completed 
a teacher preparation program focused on content knowledge, pedagogical course work (including learning theories, 
developmental theories, theories of motivation and issues of student assessment) and practice teaching. Although 
variations in the philosophy, implementation and quality of teacher education programs are enormous, research 
nevertheless suggests that many versions of this form of preparation are successful in providing adequately trained 
teachers for the complexity of classroom instruction (Ashton & Crocker, 1987; Darling-Hammond, 1992; Wilson, 
Floden, & Ferrini-Mundy, 2002). 



McDiarmid and Wilson's (1991) research is relevant to this point. They demonstrated that subject matter knowledge 
is not sufficient to prepare teachers for teaching the concepts in these fields to students. They did this by evaluating 
mathematics majors in alternative certification programs that stressed subject matter knowledge and found that those 
teachers had strongly held misconceptions about the subject matter and the appropriate ways to teach the content. 
Their results indicated that these teachers were unable to integrate their subject matter knowledge with teaching 
practices to allow for effective instruction. In effect, because they were lacking in education coursework, they were 
unable to provide the appropriate instruction to students. 

Another aspect of good quality certification programs is that they provide experiences for the preservice teachers in 
classrooms both under direct supervision and with continued mentoring. Darling-Hammond (1992) notes that skills 
need to be learned in context, where they can be practiced under supervision. The student teaching experience allows 
the preservice teacher to transform information from coursework in order to learn its character in the context of the 
real world of teaching in classrooms. Jelmberg (1996), cited above, compared traditionally certified teachers with 
alternatively certified teachers' performance based on their experience. His results showed that experienced teachers 
from traditional certification programs are rated higher on instructional skills and planning by their principals, and 
perform better, than did experienced teachers who came from alternative certification programs. 

Emergency certification 

The reviews of research, above, compared alternative certification programs, some of which provide enough 
preparation for graduates to receive full certification, while others provide minimal training resulting in graduates 
receiving either a provisional or emergency certificate. Research reviewed above also compared fully certified 
teachers to one another, distinguishing between teachers who taught in-field or out-of-field. There is little research 
comparing emergency certified teachers and regularly certified teachers. 

Emergency certificates are issued to prospective teachers who have met some, but not all of the requirements for state 
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certification. Minimum requirements are often a bachelor's degree and a passing score on a certification exam. 
Emergency certificates are issued for a limited time period, usually one to two years. Some states allow for these to 
be renewed, while others states issue a one time only, nonrenewable certificate. In 1998 data about certification 
waiver rates were available from 39 states (U.S. Department of Education, 1999). Sixteen of the 39 states had waiver 
rates greater than 2% of their teaching population, eight had rates over 5% while some had rates as high as 17%. 



Emergency certified teachers are more likely to be hired in already low performing schools, schools that serve low 
SES students, schools in rural and inner city areas, and for positions that are hard to fill (Darling-Hammond, 1997a; 
Darling-Hammond, 2001; Ingersoll, 2001 ; Gitomer, Latham, & Ziomek, 1999). Since high poverty schools are more 
likely to have high rates of out-of-field teaching as well as more emergency certified teachers, it may that some states 
are failing to provide the "adequate education" that most state constitutions require (Ingersoll, 2001 ; Hirsh, Koppich 
& Knapp, 2001 ; Rivkin, et. al., 2000). And now that federal government has waded in on this issue, requiring a 
highly qualified teacher in every classroom, the competency of teachers with emergency certification is sure to be 
questioned further. 



Emergency certification is justified on the basis of three arguments. First, it is argued, that there is a teacher shortage 
requiring that states emergency certify teachers to provide enough teachers for every classroom. But the shortage 
may not be as severe as many claim (Feistritzer, 1994; Ingersoll, 1997, 2001; Ballou, 1996; Hirsh, Koppich, & Knapp, 
2001; Hardy, 1998). The National Center for Education Statistics indicates that the teaching force will grow to more 
than 3 million teachers by the year 2008 (US Department of Education, 1999). But it is a misconception that colleges 
of education will need to train millions of new teachers to meet the needs of school districts. Darling-Hammond 
(cited in Hardy, 1998) believes that this potential shortage is not universal, claiming that "there are districts that 
experience difficulty hiring qualified teachers, but overall, we have a surplus of teachers" (p.20). The teacher 
shortages are seen in subject areas like mathematics and science; in the service areas for special needs populations, 
such as special education and bilingual education; and shortages exists in rural areas and in inner city school districts 
(Wayne, 2000; Natriello & Zumwalt, 1993; Hardy, 1998; Hirsch, Koppich, & Knapp, 2001). 



The projected teacher shortage is also based on assumptions of increased student enrollments and an aging workforce. 
But these assumptions have similarly been questioned. Research evaluating the Survey of Recent College Graduates 
(Ballou, 1996) has shown that: 



In every year there were at least twice as many [qualified] applicants as there were persons hired in 
full-time public school positions. Far from indicating that the nation faces a teacher shortage, these 
data show that the teacher labor market as a whole has been in a chronic state of excess supply, 
though shortages may arise in specific locations and subject areas (p. 101). 



Research also indicates that regularly certified teachers are in short supply because of poor pay; low levels of job 
satisfaction, particularly when working with disadvantaged minority students (Hanushek, Kain & Rivkin, 2001); and 
because limited faculty input about the management of schools discourages college graduates from teaching and 
drives current teachers out of the profession. Ingersoll (2001) suggests that: 

The imbalance of teacher supply and demand at the root of school staffing problems is neither 
synonymous with, or primarily due to, teacher shortages in the technical sense of a deficit in the 
quantity of qualified candidates. Rather than insufficient supply, the data indicate that school staffing 
problems are primarily due to excess demand, resulting from a 'revolving door' — where large 
numbers of teachers depart their jobs for reasons other than retirement. Thus the solution.. .does not 
primarily lie in increasing supply, but rather in decreasing demand (p. 501). 



A second argument for emergency certification is that there are many people who would teach, but do not, because 
standard certification requirements prohibit them from doing so. Thus opponents of traditional teacher education 
programs call for the removal of certification requirements, claiming that there is no "special body of knowledge" that 
teachers need to know in order to be successful. These advocates for the abolition of requirements claim that what 
needs to be learned by new teachers can be learned in the first year of teaching (Roth, 1 994; Kanstroom & Finn, 1 999; 
Finn & Kanstroom, 2000). In fact, anecdotal evidence claiming that teachers learn more in the first year of teaching 
than they do in their education courses is quite strong. Armed with this knowledge, it is then argued that a person 
who holds a college degree, in possession of some level of content knowledge, and with some limited experience 
teaching youth, is competent enough to begin to teach. Such beliefs drive the movement against certification despite 
research that argues against this position (McDiarmid & Wilson, 1991). 




Traditional certification programs are rejected by many adults who would be interested in teaching as a second career 
but who will not or cannot engage in time consuming and expensive regular teacher certification programs. 
Proponents of alternative certification believe that these adults, called career transitioners, have skills that they have 
learned in their other employment that could be used to teach children (MacDonald, Manning, & Gable, 1994). 
Additionally, some believe that adults have unique life skills and experiences that can be useful to students (Zumwalt, 
1991). Research does indicate that alternative certification programs attract an older and more diverse population, 
through their more flexible schedules, less stringent requirements, and so forth; however, it is unclear from this 
research that certification needs to be waived in order to recruit a more diverse teaching population with many life 
skills and employment experiences (Natriello & Zumwalt, 1993; Bradshaw & Hawk, 1996; Hawley, 1992). 
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The third argument for emergency certification makes use of the long-standing lack of confidence by state officials 
and the general public in the quality of the teachers who graduate from colleges of education. Too often colleges of 
education are perceived to attract less able students, thus producing under qualified teachers. This is simply not true. 
Research supports the assertion that the academic quality of students entering colleges of education is quite good. For 
example, Gitomer, Latham, and Ziomek (1 999) showed in their analysis of ACT and SAT scores that students from 
colleges of education were as academically skilled as students with other college majors. They also reported that 
traditionally certified teachers have the highest passing rates on certification exams (PRAXIS I and II) compared to 
alternatively certified and emergency certified teachers, even though they appeared to be similar in initial 
achievement, based on SAT scores. They concluded that traditional certification (having training in teaching methods, 
pedagogy and practice in teaching) makes a difference on licensure. They attribute the better performance of 
traditionally certified teachers to the training and instruction, provided by colleges of education. 

Summary of Related Research 

With regard to teacher subject matter the current research suggests to us that in mathematics, especially, and at the 
upper grades, in particular, subject matter competency as assessed by college majors, courses taken, and degrees held, 
leads to higher student achievement. Professional pedagogical knowledge appears equally important a contributor to 
student achievement at the upper grades, and may even be more important than content knowledge in the elementary 
grades. 

With regard to experience through teacher education course work and by means of learning on the job, the research 
suggests that student achievement is affected in positive ways. The powerful effects of content knowledge, 
pedagogical knowledge, and experience, contribute to success in teaching in one's own field. When teaching out-of- 
field, such knowledge and skill is of less use and teacher effectiveness is compromised. 

When we examined alternative routes to teaching we found them to be quite variable. Still, in comparison to 
traditional teacher education programs (that are also quite variable) the alternative programs take less time, are less 
costly, attract more diverse students, but also record higher drop-out rates. Because they usually take much less time, 
alternative programs may suffer from under-preparing students for the classroom, a problem compounded by the lack 
of supervision and support given by the hiring districts. Teach for America, as one alternative route to teaching, 
appears to suffer from the ordinary and typical problems inherent in the designs of such programs. While criticisms 
abound, there is a substantial body of literature suggesting that traditional teacher education programs, warts and all, 
seems to provide more competent appearing teachers whose students perform better. 

The research suggests that emergency certified teachers are probably the least prepared to do well. Unfortunately, 
such teachers are typically hired into districts with the hardest to teach students. 

After reviewing this literature it seemed as if the conditions necessitating out-of-field teaching and the hiring of 
alternate and emergency certified teachers could easily be modified, eliminating the need for these practices to exist. 

In the past, however, neither politicians nor school boards had the will to do that. But under the new federal No Child 
left behind Act of 2001, school districts will have to have a "highly qualified" teacher in every classroom or loose 
federal funding. It will be an interesting few years as ways to define "highly qualified" and the related term 
"competent" are worked on to meet the letter of the law. The research in these areas is also likely to be reinterpreted in 
light of that law. In the near future, surely both the definitions of, and the research associated with the idea of teacher 
quality, will find their ways into courtrooms of America. This study may help the courts in thinking about what it 
means to have a highly qualified teacher in the classroom. 

Method 



Research Design 



An ex-post-facto archival research design was used to study the performance of students in the classes of the under- 
certified and certified teachers in selected districts of the state of Arizona. Districts provided both the information 
about the teachers participating in this study and their Stanford Nine (SAT 9) class means. The SAT 9 data provided 
by districts were also compared with the same data obtained from the Arizona Department of Education. SAT 9 data 
were available from 1997-2000, but was not available for the 2000-2001 academic year. 

Sample 

In Arizona under-certified teachers have three labels: "emergency" (for holders of bachelor degrees from accredited 
institutions, with little or no educational coursework, who can get clearance of criminal background through 
fingerprint analysis); "temporary" (a rarely used designation similar to "emergency"); and "provisional" (for those 
with some, or even considerable teacher education training, who are short certain units or requirements that could earn 
them a standard certificate). In opposition to the under-certifed teachers are the regularly certified teachers who meet 
all of the state requirements for certification. These teachers hold a bachelor’s degree from an accredited institution, 
have completed 45 semester hours of elementary or secondary education course work, obtained a passing scores on 
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the AEPA, demonstrated that they understand both the Arizona and US constitution, and been vetted for any criminal 
background through analysis of their fingerprints. (The definitions of under-certified and certified teachers, given 
above, were accurate through the year 2000, when the Arizona Department of Education made changes to the 
certification laws. Certification requirements are still in flux, and so current Arizona certification policies may not be 
the same as those reported in this study.) 

Among the under-certified teachers in this study are some from an alternative route to traditional teacher training, the 
program "Teach for America (TEA)." Teach for America is a popular alternative certification program. Its mission 
calls for placing energetic, bright, but unqualified teachers into poor, urban school districts (Darling-Hammond, 

1 994). The instructional effectiveness of under-certified teachers in general, and the TEA teachers in particular, is of 
considerable interest to the policy community. 

District Selection, Arizona school district superintendents listed by the department of education as participating in 
the Eederal Teacher Shortage Loan Deferment Program (Arizona Department of Education, 
http://ade.az.gov/researchpolicy/ts) were invited to participate in this study. This federal program requires the State 
Department of Education to generate a ranking of school districts in the state by the percent of under-certified 
teachers in each district. This list of school districts provided a convenient population from which to obtain a sample. 
Of the school districts receiving a request to participate (N=24 for the 1998-1999 dataset, N=12 for the 1999-2000 
dataset) five school districts responded positively. These five school districts represent 20,8% and 16.6%, 
respectively, of the school districts designated by the state as having severe teacher shortages. All five school districts 
chosen for this study were included in the Department of Education’s 1998-1999 classification of school districts. 

Only two of these school districts were also included in the Department of Education's 1 999-2000 classification of 
school districts with severe shortages. All of the participating school districts shared similar characteristics. They all 
serve inner city student populations, largely minority, and all participate in the Teach for America (TEA) program. 

All the schools in these districts have difficulty filling teaching positions. 

Methods of data collection. The five participating school districts provided lists of new hires for the 1998-1999 and 
1999-2000 school years. In some cases this list contained information about the teachers' certification status, while in 
other cases further research was necessary to obtain this information. We were granted permission to access 
individual teacher personnel files in order to collect the necessary data on the school where the teacher was employed, 
the grade level taught, the teachers' certification status, their highest degree earned, the date and institution where 
there degree was earned, their age (determined from year of their birth), and teaching experience. 

Of the teachers included in the dataset, 64% had no prior teaching experience, a judgment based on their hire date, 
employment history, resume and application. The majority of newly hired teachers were recent graduates from 
college. Some of the new teachers, however, had delayed their entrance into teaching for many years after 
graduation, but they had no prior teaching experience indicated in their personnel files. Teachers were removed from 
the sample if they taught kindergarten, first grade, art, music or special education, grades and subjects not assessed by 
the Stanford Nine (SAT 9). 

The assessment departments of each school district provided test scores aggregated at the classroom level. Included 
were the teachers' class totals as raw scores, scaled scores, grade equivalent, national percentile rank, stanine, national 
normal curve equivalent, as well as class percentile rank and class stanine. Individual student scores were not needed 
or provided. Additionally, state level SAT 9 data was later obtained from the Research and Policy division of the 
State Department of Education in order to confirm the accuracy of the SAT 9 data collected from the school districts. 
State level data was aggregated, by teacher name, for each of the school districts. In the event of a discrepancy 
between the two sets of data, we opted to use the state level data. This discrepancy occurred with only one of the 
school districts data files. 

Matching Procedure 

In the five districts studied, 293 teachers were hired in 1998-1999 and 1999-2000 who met the requirements for 
inclusion in this study. In order to participate teachers personnel files were required to contain the demographic data 
necessary for analysis, as well as classroom level SAT 9 scores. Teachers' data files were matched using SPSS 
procedures to sort the files. Teachers in each district were first categorized on the basis of their certification status, 
under-certified teachers (labeled emergency, provisional or temporary certified teachers by their districts) constituted 
one group, certified teachers made up the other group. Teachers from each group were then matched based on grade 
level and highest degree attained. Teachers for whom no matches could be found were removed from the analysis. 

Matches were made using the following rules: 1) matches were first made within the same school, 2) matches were 
made within the same school district, and 3) matches were made between similar school districts. The first and 
second matching rules serve to minimize exogenous variables associated with student achievement scores, e.g., socio- 
economic status, school characteristics, curriculum, etc. It is assumed that teachers in the same school teach similar 
students, an imperfect but reasonable assumption. The identical assumption can be made about schools within the 
same district boundaries, since Arizona school district boundaries are based on relatively homogenous geographic 
areas. Conversations with district personnel, in the course of collecting the data at the district offices, provided no 
evidence that the certified or under-certified teachecs were "tracked" in any way. The assignment of teachers to 
schools, and to classrooms within schools, appears to have been unbiased. Similarly, we have no reason to believe 
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that class size or student ability was different in any way for the certified or under-certified teachers in our sample. 

The cross-district matching of teachers, however, is more problematic than the within district matching. We made 
these matches based on the "sameness" of the two districts. Sameness was determined using data collected from the 
Education Finance Statistics Center, a subdivision of the National Center for Education Statistics (U.S. Department of 
Education, 2001). Using public school district financial records for the latest year available, 1996-1997, data about 
each of the participating school districts were collected for: student teacher ratio, administrative ratio, median income, 
percent of householders with high school graduates, percent of non-white children, percent of limited English 
proficiency, and percent of children in poverty. School districts that shared similar characteristics were matched to 
one another. This procedure matched the Osborn Elementary School District to the Creighton Elementary School 
District, and matched the Roosevelt Elementary School District to both the Nogales Elementary School District and 
the Murphy Elementary School District. These are not pseudonyms, but the actual names of the Arizona school 
districts that generously helped us in this study. 

Synthesizing data from the various state departments of education, GreatSchools.net also provided data that were used 
to match schools having similar characteristics. Sameness matches were also made based on: AIMS reading and math 
scores, SAT 9 reading and math scores, teacher experience, percent of teachers with masters degrees, attendance rate, 
open enrollment, percent free and reduced lunch, and ethnic breakdown within the school district. These data support 
the matches that were made using the NCES finance data with the exception that the Murphy School District was 
found to more closely match the Creighton School District. Teacher matching across districts, therefore, was 
accomplished by finding similar school districts serving similar student populations with similar economic bases. The 
assertion that teachers in these different school districts are sensibly matched is well substantiated, but we 
acknowledge that the nature of the matching procedures used constitutes a potential flaw in studies such as this. 
Random assignment of under-certified and certified teachers to classes within districts would have provided a stronger 
design, but this was not possible. It is important to note, however, that the matches in this study were made with out 
any knowledge of the teachers' class scores on the SAT 9. 

Descriptive analyses were conducted on the complete sample of teachers in the data set (N= 293). These results are 
provided in Table 1 . 



Table 1 

Descriptive Statistics for 1998-1999 and 1999-2000 Datasets 





Certified 
Teachers N=159 


Emergency Certified 
Teachers N=89 


Temporary Certified 
Teachers N=19 


Provisionally Certified 
Teachers N=26 


District 


Murphy 


21 


13 


10 


1 


Creighton 


64 


34 


0 


0 


Osborn 


21 


2 


2 


2 


Roosevelt 


41 


35 


19 


19 


Nogales 


12 


5 


4 


4 


Grade 


2 


25 


9 


3 


7 


3 


19 


8 


2 


5 


4 


29 


7 


2 


4 


5 


24 


13 


1 


2 


6 


21 


22 


4 


5 


7 


22 


14 


3 


2 


8 


19 


16 


4 


1 


College 


ASU 


74 


13 


6 


9 


Uof A 


9 


6 


1 


2 


NAU 


16 


5 


3 


3 


U of Phoenix 


0 


4 


0 


0 


Ottawa 


8 


0 


2 


1 


Grand Canyon 


0 


1 


0 


1 


Out-of-State (large 


14 


25 


2 


4 



public) 










Out-of-State (small 
public) 


13 


8 


1 


1 


Out-of-State (large 
private) 


4 


7 


0 


0 


Out-of-State (small 
private) 


6 


8 


2 


4 


Foreign Education 


9 


10 


0 


0 


Prescott College 


1 


2 


2 


0 


Other 


4 


0 


0 


0 




Certified 
Teachers N=159 


Emergency Certified 
Teachers N=89 


Temporary Certified 
Teachers N=19 


Provisionally Certified 
Teachers N=26 


Major 


Education 


9 


7 


1 


0 


Elem. Ed. 


74 


7 


14 


19 


Second. Ed. 


5 


1 


0 


0 


Early Child. 


4 


0 


0 


0 


C&I 


6 


0 


0 


0 


Ed. Admin. 


6 


0 


0 


0 


Bilingual Ed. 


8 


0 


2 


0 


Spec. Ed. 


2 


0 


0 


0 


Phys. Ed. 


1 


0 


1 


0 


Liberal Arts 


10 


12 


0 


0 


Soc. Sciences 


23 


25 


11 


4 


Sciences 


4 


17 


0 


3 


Business 


4 


7 


0 


0 


Languages 


1 


11 


0 


0 


Other 


2 


2 


0 


0 


Degree 


BA 


126 


77 


17 


25 


MA 


32 


12 


2 


1 


PhD 


1 


0 


0 


0 


TFA Status 


Yes 


0 


32 


0 


2 


No 


159 


57 


19 


24 



Note: N = 293 



From this population, teachers for whom no matches were found were removed from the analysis. The initial 
matching procedures produced 1 16 pairs of teachers (N=232 individual teachers) out of the total teacher pool of 293, 
thereby using 79% of the original dataset. We undertook analyses of the quality of the matching procedures that were 
used. These are reported on, below, and each analysis is based either on the entire sample drawn (N = 293), or on the 
pairs that were created by the matching procedure that was used (N = 232). 



However, the process of matching produced 28 mismatched pairs consisting of teachers who did not share class score 
data for the same test administrations. This occurred, for example, when one teacher in a pair had scores for 1998 and 
the other teacher had scores for 1999. Additional matches were then made based on all of the above matching rules, 
but eliminating cross year matches. This finally resulted in N=I09 matched pairs, using 74% of the original data set. 
The appendix to this study contains descriptive information about the 109 pairs of teachers who comprise the 
sample for this study. The data are also available as a Microsoft Excell spreadsheet. 

Matching in primary schools across grades 3-8, however, created a problem inherent in archival studies such as this 
one. To design this study sensibly we needed teachers of self-contained classes. If departmentalization (more than 
one teacher working with the class) were occurring we would have problems inferring a teacher’s affect on student 
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learning. But we have no knowledge of what went on in every school at these upper grade levels. We were told, 
however, that these schools used little departmentalization, and that the teachers for whom we had files were the' 
classroom teachers of record for the district and the state. The 218 teachers in our sample were, therefore, the 
teachers designated by administrators as those responsible for their student ’s achievements on the SAT 9 tests. Since 
these were the responsible teachers we included all the matches from grades 3 to 8. Nevertheless, because we worried 
about the issue of departmentalization in the upper two grade levels, we ran separate analyses. One set of analyses 
was done with the full sample of 109 pairs of certified and under-certified teachers, and another set of analyses was 
done with a reduced sample, eliminating all the pairs of teachers in the 7th and 8th grade. The appendix to this report 
describes all 1 09 pairs of teachers by grade level and thus identifies which pairs were eliminated from the second 
analysis. In the discussion that follows, when the second analysis using the smaller sample of pairs of teachers from 
grades 3-6 produces results different from the analysis of the entire sample, we will note these differences. 

Instruments 



The Stanford Achievement Test, Ninth Edition (SAT 9), a nationally norm referenced standardized test is used by all 
districts in the state of Arizona and was used, as well, in this study. The test assesses student achievement in reading, 
math and language arts (Harcourt Brace, 1 997). The SAT 9 is believed by the State Department of Education to relate 
to Arizona academic standards, which teachers use as a guide to instruction. It is claimed that The SAT 9 tests 
between 70*80% of the material outlined in the state’s academic standards (ADE personal communication, 2001). 

This relationship is strongest for the reading and mathematics subtests, and in grades 2 through 8. The language 
subtest of the SAT 9 is not as well related to state standards because it does not require a writing sample of students, 
an ability that is promoted in the standards. The analysis of the State Department of Education suggests that the SAT 
9 is a reasonable indicator of student achievement, perhaps more for reading and mathematics than for language. 
Furthermore, in Arizona the test is often used as an indicator of teacher and school effectiveness. 

Scoring. Once teachers were matched to one another their Stanford Achievement Test, Ninth Edition (SAT 9) scores 
were aggregated at the class level. For each pair of teachers their mean National Normal Curve Equivalent (NCE) 
scores for SAT 9 reading, math and language were analyzed. The NCE scores are a type of normalized standard score 
resulting form the division of the normal curve into 99 equal units. This score is traditionally used for research 
purposes, enabling researchers to interpret differences in NCEs more readily because of the equal-interval nature of 
the NCE scores. Differences between NCEs obtained by different groups have the same meaning regardless of what 
part of the scale is referenced. For the purposes of this study individual student scores were not collected and thus 
cannot be reported. 

Analyses of Matching Procedures 

The credibility of the matching procedure for pairing uncertified and certified teachers is important in the 
interpretation of the results of this study. Therefore, we undertook some analyses to explore that issue. We began by 
looking at the similarity of the SAT 9 test scores in each school and district to determine their comparability to each 
other, a check on the level of ’’sameness” of each school and district to one another. These analyses were conducted 
using NCE scores on the mathematics, reading and language sub-tests of the SAT 9, for both the 1998-1999 and 
1999-2000 data sets. Alpha levels were set at p= .05. 

Matching Analysis /. To answer the question about whether students’ test scores on the SAT 9 are different as a 
function of which school they attend, a one-way analysis of variance (ANOVA) was conducted on the entire data set 
(N=293) to evaluate the relationship between the school assignments and student achievement scores. The 
independent variable was teachers’ school assignment; the dependent variable was mean student achievement scores 
of these teachers as measured in Normal Curve Equivalents (NCE) for reading, mathematics and language in 1998- 
1999 and 1999-2000. The ANOVA was not significant with the exception of mathematics scores in the 1999 sample. 
These results are provided in Table 2. The results indicate that, overall, the schools from whom teachers in the 
sample came showed no statistically significant differences from each other in terms of mean NCE scores on the SAT 
9, except for the special case of mathematics scores in the 1999 sample, F (4 1,1 90) = 1.65,p = .01. It is unlikely that 
there are inherent differences in the schools that could bias the findings of subsequent analyses. 

Table 2 

ANOVA Results for School Sameness 



SAT 9 Subtest and Year 




Sum of Squares 


df 


Mean Square 


F 


Sig. 


Reading 1998 


Between Groups 


2853.47 


36 


79.26 


1.00 


0.487 




Within Groups 


7389.23 


93 


79.45 








Total 


10242.70 


129 








Math 1998 


Between Groups 


2801.62 


36 


77.82 


1.07 


0.385 




Within Groups 


6753.20 


93 


72.62 





















22Q 





Total 


9554.82 


129 








Language 1998 


Between Groups 


3887.73 


36 


107.99 


1.20 


0.237 




Within Groups 


8341.16 


93 


89.69 








Total 


12228.89 


129 








Reading 1999 


Between Groups 


4621.26 


41 


112.71 


1.42 


0.060 




Within Groups 


15037.23 


190 


79.14 








Total 


19658.49 


231 








Math 1999 


Between Groups 


6176.95 


41 


150.66 


1.65 


0.014* 




Within Groups 


17398.93 


190 


91.57 








Total 


23575.88 


231 








Language 1999 


Between Groups 


4831.13 


41 


117.83 


1.37 


0.083 




Within Groups 


16363.21 


190 


86.12 








Total 


21194.3404 


231 









Note. * Indicates significance p= .05 
To determine df for 1998-1999 sample: 

BG= each school with cases N= 37, df = 37-1= 36 
WG= each case (130) - total groups (37) df = 130-37=93 
Total df=N-l, 130-1 = 129 
To determine df for 1999-2000 sample: 

BG = each school with cases N = 42, df = 42-1= 41 
WG= each case (232) - total groups (42) df = 232-42 = 190 
Total df=N-l, 232-1 =231 

Matching Analysis 2. To answer the question about whether the test scores of students whose teachers might be 
paired in later analyses differed as a function of which district they attend, a one-way analysis of variance (ANOVA) 
was conducted on the entire data set (N=293) to evaluate the relationship between the district assignments and student 
achievement scores. The independent variable was teachers' district assignment; the dependent variable was the 
classroom mean student achievement scores of these teachers as measured in Normal Curve Equivalents (NCE) for 
reading, mathematics and language in 1998-1999 and 1999-2000. The results indicate significant differences between 
the mean NCE scores for all subtests and for both years of data. For the 1 998-1999 data set the ANOVA was 
significant, F (4, 126) = 3.20, p= .02 for reading, F (4,1 26) = 2.81,p= .03 for mathematics, and F (4,1 26) = 3.38, 
p= .0 1 for language. For the 1 999-2000 data set, F (4, 228) = 8. 1 9, p= .0 1 for reading, F (4, 228) = 8.75, p= .0 1 for 
mathematics, and F(4, 228)= 6.93, p= .01 for language. These results are provided below in Table 3. 

Table 3 

ANOVA Results for District Sameness 




SAT 9 Subtest and Year 




Sum of Squares 


df 


Mean Square 


F 


Sig. 


Reading 1998 


Between Groups 


957.57 


4 


239.39 


3.20 


0.015* 




Within Groups 


9413.53 


126 


74.71 








Total 


10371.10 


130 








Math 1998 


Between Groups 


799.17 


4 


199.79 


2.81 


0.028* 




Within Groups 


8964.88 


126 


71.15 








Total 


9764.05 


130 








Language 1998 


Between Groups 


1183.68 


4 


295.92 


3.38 


0.011* 




Within Groups 


11045.56 


126 


87.66 








Total 


12229.24 


130 








Reading 1999 


Between Groups 


2471.10 


4 


617.78 


8.19 


.000* 




Within Groups 


17200.50 


228 


75.44 








Total 


19671.60 


232 








Math 1999 


Between Groups 


3144.95 


4 


786.24 


8.75 


.000* 




Within Groups 


20486.51 


228 


89.85 








Total 


23631.46 


232 
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Language 1999 


Between Groups 


2297.24 


4 


574.31 


6.93 


.000* 




Within Groups 


18902.61 


228 


82.91 








Total 


21199.85 


232 









Note. * Indicates significance /?= .05 
To determine df for 1998-1999 sample: 

BG= each district N= 5, df= 5-1=4 

WG= each case (131) - total groups (5) df= 131-5=126 

Total df=N-l, 131-1= 130 

To determine df for 1999-2000 sample: 

BG= each district N= 5, df= 5-1= 4 

WG= each case (233) - total groups (5) df= 233-5=228 

Total df=N-l, 233-1= 232 



The results of this ANOVA indicate that at the district level the mean student NCE scores were statistically different 
from one another for the teachers who comprise the population from which our sample would be analyzed. This 
suggests that the procedures we used to match teachers across districts were not faultless. But analysis of the mean 
student scores across districts suggests that only one district may have been an outlier, with slightly higher SAT 9 
scores than the others. Since only 38% of all teachers had to be matched with teachers from another district, it is 
likely, therefore, that only a small percent of those matches could have been problematic, totaling less than 10% of all 
the matches that were made. In addition, the matching of teachers across district lines was based on multiple 
measures of district sameness; NCE scores provide only one such measure. Because of that, we believe that the 
matching of teachers across district lines can still be defended as a reasonable way to obtain a sample for analysis of 
the student achievement of certified and uncertified teachers. 

The discrepancy between the results for the ANOVA on the participating schools and the participating districts is 
curious and remains an issue to be resolved. This is, of course, one of the reasons that hierarchical designs have 
become necessary in the analysis of classroom, school and district level data. But for the purposes of this study, it is 
not clear that this discrepancy would cause any systematic bias in the data analyses to follow. 



Matching Analysis 3. After the 1 09 pairs of matched teachers were identified we then inquired whether the average 
SAT 9 scores of certified teachers differed as a function of whether they were matched with teachers within their 
same school district or with teachers from another participating school district. A one-way analysis of variance 
(ANOVA) was used in this analysis. The independent variable was teachers' district assignment, either within or 
between school district. The dependent variable was the student achievement scores of these teachers as measured in 
Normal Curve Equivalents (NCE) for reading, mathematics and language in 1998-1999 and 1999-2000. ANOVA 
results indicate no significant differences between the mean NCE scores for the certified teachers matched within the 
same district as compared to certified teachers matched between districts. These results are provided below in Table 
4. 



Table 4 

ANOVA Results for Certified Teacher Matches 
Within and Between Districts 



SAT 9 Subtest and Year 




Sum of Squares 


df 


Mean Square 


F 


SIg. 


Reading 1998 


Between Groups 


66.66 


1 


66.66 


0.84 


0.36 




Within Groups 


4195.26 


53 


79.16 








Total 


4261.92 


54 








Math 1998 


Between Groups 


56.29 


1 


56.29 


0.76 


0.39 




Within Groups 


3932.17 


53 


74.19 








Total 


3988.46 


54 








Language 1998 


Between Groups 


8.88 


1 


8.88 


0.11 


0.74 




Within Groups 


4251.57 


53 


80.22 








Total 


4260.45 


54 








Reading 1999 


Between Groups 


30.45 


1 


30.45 


0.36 


0.55 




Within Groups 


7566.44 


89 


85.02 








Total 


7596.89 


90 








Math 1999 


Between Groups 


4.68 


1 


4.68 


0.05 


0.82 




Within Groups 


8336.88 


89 


93.67 
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Total 


8341.56 


90 








Language 1999 


Between Groups 


23.20 


1 


23.20 


0.33 


0.57 




Within Groups 


6279.82 


89 


70.56 








Total 


6303.02 


90 









To determine df for 1998-1999 sample: 

BG= match type N= 2, df= 2-1=1 

WG= each case (55) - total groups (2) df= 55-2=53 

Total df=N-l,55-l=54 

To determine df for 1999-2000 sample: 

BG= match type N= 2, df= 2-1= 1 

WG= each case (91) - total groups (2) df= 91-2=89 

Total df=N-l, 91-1= 90 



Matching Analysis 4. After the 109 pairs of matched teachers were identified we then inquired whether the average 
SAT 9 scores of under-certified (emergency, temporary or provisional certified) teachers' differed as a function of 
whether they are matched within the same school district or with teachers from another participating school district. 

A one-way analysis of variance (ANOVA) was conducted. The independent variable was teachers' district 
assignment, either within or between school district. The dependent variable was the student achievement scores of 
these teachers, as measured in Normal Curve Equivalents (NCE) for reading, mathematics and language in 1998-1999 
and 1999-2000. In general, the ANOVA results indicate no significant differences between the scores of under- 
certified teachers matched within district as compared to under-certified teachers matched between districts. But that 
was not true for all tests. Significant differences in NCE scores were found for mathematics in the 1999-2000 data 
set. For mathematics, F (1 , 93)= 8.08, /? = .01 . The exclusion of 7th and 8th grade teachers yielded similar results; F 
(1, 67)= 4.12,/? = .047. These results are provided below in Table 5. 

Table 5 

ANOVA Results for Under-Certified Teacher Matches 
Within and Between Districts 



SAT 9 Subtest and Year 




Sum of Squares 


df 


Mean Square 


F 


Sig. 


Reading 1998 


Between Groups 


0.93 


1 


0.93 


0.02 


0.90 




Within Groups 


2214.40 


35 


63.27 








Total 


2215.33 


36 








Math 1998 


Between Groups 


19.52 


1 


19.52 


0.45 


0.51 




Within Groups 


1503.55 


35 


42.96 








Total 


1523.07 


36 








Language 1998 


Between Groups 


14.09 


1 


14.09 


0.17 


0.68 




Within Groups 


2936.85 


35 


83.91 








Total 


2950.94 


36 








Reading 1999 


Between Groups 


69.52 


1 


69.52 


0.82 


0.37 




Within Groups 


7849.48 


93 


84.40 








Total 


7919.00 


94 








Math 1999 


Between Groups 


688.59 


1 


688.59 


8.08 


0.01* 




Within Groups 


7928.94 


93 


85.26 








Total 


8617.53 


94 








Language 1999 


Between Groups 


246.32 


1 


246.32 


3.27 


0.07 




Within Groups 


6856.38 


91 


75.35 








Total 


7102.71 


92 









Note. * Indicates significance p= .05 
To determine df for 1998-1999 sample: 

BG= match type N= 2, df= 2-1=1 

WG= each case (37) - total groups (2) df= 37-2=35 

Total df=N-l, 37-1= 36 

To determine df for 1999-2000 sarnple: 

BG= match type N= 2, df= 2-1=1 
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WG= each case (95) - total groups (2) df= 95-2=93 

Total df=N-l,95-l=94 

For 1999-2000 Language: Total cases= 93 



It has been argued, above, that the matching procedures used in this study were sensible. The four statistical analyses 
intended to evaluate the matching procedures provide evidence that they were not perfect, but that evidence does not 
lead to the conclusion that the approach taken in this study was unreasonable or would lead to faulty conclusions. The 
matching of the pairs of teachers, one certified with one under-certified, within and across district lines, took place 
before the SAT scores of the teachers in each pair were scrutinized. Thus the matching procedures appear to be 
unbiased with regard to the research questions that are of interest. The results of the analyses appropriate to these 
research questions are considered next. 



Results 



We first chose to look at whether the three kinds of under-certified teachers differed among themselves. We believed 
that subsequent analyses would be simpler if the SAT 9 NCE scores of the students of these three groups of teachers 
were not statistically different from each other. If that were the case, we could treat the three sub-groups of under- 
certified teachers as a single group. 

A one-way analysis of variance (ANOVA) was conducted in which the independent variable was teachers’ 
certification, while the dependent variable was the student achievement scores of these teachers as measured in 
Normal Curve Equivalents (NCE) for reading, mathematics and language in 1998-1999 and 1999-2000. Results 
indicate that the NCE scores for all under-certified teachers (emergency, temporary and provisional) were not 
statistically different from one another. These results are provided below in Table 6. Because of this finding we 
subsequently treated all uncertified teachers as a homogeneous group. In all subsequent analyses we will compare 
certified teachers to the total group of under-certified teachers (emergency, temporary and provisional). 



Table 6 

ANOVA Results for Emergency, Temporary & Provisional 
Certified Teachers 



SAT 9 Subtest and Year 




Sum of Squares 


df 


Mean Square 


F 


Sig. 


Reading 1998 


Between Groups 


16.91 


2 


8.46 


0.13 


0.88 




Within Groups 


2198.42 


34 


64.66 








Total 


2215.33 


36 








Math 1998 


Between Groups 


68.48 


2 


34.24 


0.80 


0.46 




Within Groups 


1454.59 


34 


42.78 








Total 


1523.07 


36 








Language 1998 


Between Groups 


176.61 


2 


88.31 


1.08 


0.35 




Within Groups 


2774.33 


34 


81.60 








Total 


2950.94 


36 








Reading 1999 


Between Groups 


116.51 


2 


58.25 


0.69 


0.51 




Within Groups 


7802.49 


92 


84.81 








Total 


7919.00 


94 








Math 1999 


Between Groups 


131.56 


2 


65.78 


0.71 


0.49 




Within Groups 


8485.97 


92 


92.24 








Total 


8617.53 


94 








Language 1999 


Between Groups 


120.69 


2 


60.35 


0.78 


0.46 




Within Groups 


6982.02 


90 


77.58 








Total 


7102.71 


92 









To determine df for 1998-1999 sample: 

BG= match type N= 3, df= 3-1=2 

WG= each case (37) - total groups (3) df= 37-3=34 

Total df=N-l,37-l=36 

To determine df for 1999-2000 sample: 

BG= match type N= 3, df= 3-1=2 

WG= each case (95) - total groups (3) ^f= 95-3=92 



Total df=N-l,95-l=94 

For 1999-2000 Language: Total cases= 93 

To answer the key question in this study, whether students taught by certified teachers outscore students taught by 
under-certified teachers, a correlated /-test was conducted to evaluate the difference in student achievement scores of 
the certified and under-certified teachers. The results indicate that for 1998-1999, students taught by certified 
teachers outperformed students taught by under-certified teachers. More specifically, the reading scores of the 
students of certified teachers were significantly higher (Af=36.52, SD= 9.59) than were the reading scores obtained by 
the students of under-certified teachers (Af=30.67, SD= 8.02), / (27)= 2.36, p= .0 1 . In this same year, on the language 
test, the scores of the students taught by certified teachers were significantly higher (A/=34.33, SD= 9. 17) than were 
the scores of the students taught by under-certified teachers (Af=29.89, SD= 9.82), / (27)= 1 .81,p= .04. While the 
difference between the certified and the under-certified teachers on the mathematics test were not found to be 
significant, the results were in the same direction as they were for the reading and language tests. Students taught by 
certified teachers scored higher (Af=38.80, SD= 8.77) than did the students taught by under-certified teachers 
(Af=35.82, 577= 7.32). 

Results for 1999-2000 replicated the results of the data from 1998-1999. Students taught by certified teachers 
significantly outperformed students taught by under-certified teachers on every test. In reading, the scores of the 
students of certified teachers were significantly higher (Af=35.62, 577= 9.31) than were the scores of students 
instructed by under-certified teachers (A/=32.48, 577= 9.43), t (86)= 2.43, p= .01. In mathematics, the scores of 
students of certified teachers were significantly higher (Af=39.75, 577= 9.52) than were the scores obtained by students 
of under-certified teachers (A/=35.22, 577=9.77), / (86)= 2.95 p= .001. And in the area of language, the scores of the 
students of certified teachers were significantly higher (Af=35.60, 577= 8.57) than were the scores of the students 
instructed by under-certified teachers (Af=33.47, 577= 8.90), / (84)= 1 .71, p= .05. These results are provided below in 
Table 7. The exclusion of 7th and 8th grade teachers yielded similar, and more dramatic results; the average 
difference between these two groups increased. Moreover, in this analysis of only grades 3-6, all subtests across both 
years were found to be significantly different. 



Table 7 

Correlated t- test Results Comparing 
Certified and Under-Certified Teachers 



SAT 9 Subtest 
and Year 


Mean of 
Differences 


St. Dev. 


St. Error 
of Mean 


95% Conf. Int. 
Lower Limit 


95% Conf. Int. 
Upper Limit 


t 


df 


Sig. 


Reading 1998 


5.85 


13.11 


2.48 


0.77 


10.93 


2.36 


27 


0.01* 


Math 1998 


2.97 


11.44 


2.16 


-1.47 


7.41 


1.37 


27 


0.09 


Language 1998 


4.44 


13.01 


2.46 


-0.60 


9.49 


1.81 


27 


0.04* 


Reading 1999 


3.14 


12.07 


1.29 


0.57 


5.71 


2.43 


86 


0.01* 


Math 1999 


4.53 


14.31 


1.53 


1.48 


7.58 


2.95 


86 


0.00* 


Language 1999 


2.13 


11.49 


1.25 


-.35 


4.61 


1.71 


84 


0.05* 



Note. * Indicates significance /?= .05 

To determine df for 1998-1999 sample: 

Number of matches N=28 

Total df=N-l,28-l=27 

To determine df for 1999-2000 sample: 

Number of matches N=87 

Total df=N-l,87-l=86 

For 1999-2000 Language: Total cases= 85 



The NCE scale provides a metric for evaluating the differences between certified teachers and under-certified 
teachers. Students taught by certified teachers outscored their counterparts by 6 NCE points in reading, 3 NCE points 
in mathematics and nearly 5 NCE points in language in 1 998-1999. The results are similar for 1999-2000. Students 
taught by certified teachers outscored their counterparts by 3 NCE points in reading, 5 NCE points in mathematics, 
and 2NCE points in language. Expressed as a proportion of the standard deviation of the NCE scale represented as an 
effect size, these differences range across the two years from . 14 to .28 in reading, 14 to .24 in mathematics and .09 
to .19 in language. These results are provided below in Table 8. The exclusion of 7th and 8th grade teachers from this 
analysis yielded similar, yet more dramatic results in terms of effect size. Across the two years the range of the effect 
sizes were from .19 to .38 in reading, .24 to .28 in mathematics and .14 to.33 in language. For ease of discussion it is 
appropriate to choose a summary statistic to represent these data. A reasonable way to do that is to conclude that the 
average ES across all sub-tests of the SAT 9, across both years of testing, and across analyses, is around .20. Because 
of the relationship between effect size (ES) and yearly progress on standardized tests (Glass, 2002), one could expect 
that during one academic year in the primary grades, the students of certified teachers would make approximately 2 
months more academic growth than would the students of under-certified teachers. The academic year is a 10-month 
year so the loss of two months df 2/lOths of a year is the loss incurred by students placed with under-certified 
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teachers. That is, students pay approximately a 20% penalty in academic growth for each year of placement with 
under-certified teachers. 



Table 8 

NCE Differences between Certified and Under-Certified Teachers & 
Effect Size (ES) * Ranges for 1998-1999 and 1999-2000 



SAT Sub-Test 


1998-1999 


1999-2000 


ES Range 


Reading 


6 


3 


.14-.28 


Math 


3 


5 ^ 


.14 -.24 


Language 


4 


2 


c^ 

1 

c^ 

o 



Note. * Effect sizes (ES) when using normal curve equivalencies (NCE) must be calculated with a standard deviation of 21.06 NCE 
units. 



To answer the question whether the test scores of students of teachers in the Teach for America program are different 
from the scores of students who studied with other under-certified teachers, a one-way analysis of variance (ANOVA) 
was conducted. The independent variable was TEA trained vs. all other forms of training among the under-certified 
teachers. The dependent variable was the student achievement scores of these teachers as measured in Normal Curve 
Equivalents (NCE) for reading, mathematics and language in 1998-1999 and 1999-2000. ANOVA results indicate 
that the NCE scores of TEA teachers were not statistically different from the NCE scores for other under-certified 
teachers. These results are provided below in Table 9. 



Table 9 

ANOVA Results Teach for America Teachers & 
Other Under-Certified Teachers 



SAT 9 Subtest and Year 




Sum of Squares 


df 


Mean Square 


F 


Sig. 


Reading 1998 


Between Groups 


34.00 


1 


34.00 


0.55 


0.47 




Within Groups 


2181.33 


35 


62.32 








Total 


2215.33 


36 








Math 1998 


Between Groups 


21.30 


1 


21.30 


0.50 


0.49 




Within Groups 


1501.78 


35 


42.91 








Total 


1523.08 


36 








Language 1998 


Between Groups 


261.64 


1 


261.64 


3.41 


0.07 




Within Groups 


2689.30 


35 


76.84 








Total 


2950.94 


36 








Reading 1999 


Between Groups 


92.74 


1 


92.74 


1.10 


0.30 




Within Groups 


7826.26 


93 


84.15 








Total 


7919.00 


94 








Math 1999 


Between Groups 


1.31 


1 


1.31 


.01 


.91 




Within Groups 


8616.22 


93 


92.65 








Total 


8617.53 


94 








Language 1999 


Between Groups 


19.94 


1 


19.94 


0.26 


0.61 




Within Groups 


7082.77 


91 


77.83 








Total 


7102.71 


92 









To determine df for 1998-1999 sample: 

BG= TEA or under-certified N= 2 , df= 2-1=1 
WG= each case (37) - total groups (2) df= 37-2=35 
Total df=N-l,37-l=36 
To determine df for 1999-2000 sample: 

BG= TEA or under-certified N= 2 , df= 2-1= 1 
WG= each case (95) - total groups (2) df= 95-2=93 
Total df=N-l,95-l=93 
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For 1999-2000 Language: Total cases= 93 



Given the previous analyses, in which the students of certified teachers outperformed the students of under-certified 
teachers and the students of TFA teachers scored no different than did the students of other under-certified teachers, it 
may be that the next analysis is superfluous. Nevertheless, because of the intensity of the debate about the 
performance of TFA teachers described in the literature reviewed above, we chose to inquire whether students taught 
by certified teachers outperformed students taught by Teach for America (TFA) teachers. 

A correlated Mest was used for this analysis and it indicated that for the 1999-2000 data set, in reading, the scores of 
the students of the certified teachers were significantly higher (A/=35.53, SD= 9.87) than were the scores of the 
students of TFA teachers (A/=30.51 , SD= 6.85), / (21)= 1 .87, /?= .04. In mathematics, the scores of the students of the 
certified teachers were significantly higher (A/=41 .15, SD=9.2\) than were those obtained by students of teachers 
from the TFA program (A/=35.34, SD= 7.67), r (21)=2.13 ,/t= .02. Finally, in language, the scores of the students in 
the classes of the certified teachers were significantly higher (A/=36.43, SD= 9.48) than were the scores of the 
students of teachers trained by TFA (A/=32. 1 1, SD= 8.71), r (21)= 1.79,/?= .04. 

Although the same pattern of results were found for the 1998-1999 sample, the differences between the two groups 
were not statistically significant. We believe this occurred because of the smaller sample size in the 1998-1999 
analysis. These results are provided below in Table 10. The exclusion of 7th and 8th grade teachers yielded similar, 
and more dramatic results; the average difference between these two groups increased. In this analysis the 
differences were found to be significant in both years, in all subtests, except for math in 1998-1999. 

Table 10 

Correlated t-test Results Comparing 
Certified Teachers and Teach for America Teachers 



SAT 9 Subtest 
and Year 


Mean of 
Differences 


St. Dev. 


St. Error 
of Mean 


95% Conf. Int. 
Lower Limit 


95% Conf. Int. 
Upper Limit 


t 


df 


Sig. 


Reading 1998 


4.28 


10.10 


3.57 


-4.17 


12.73 


1.20 


7 


0.13 


Math 1998 


2.25 


10.02 


3.54 


-6.13 


10.62 


0.63 


7 


0.27 


Language 1998 


2.57 


7.15 


2.53 


-3.41 


8.54 


1.02 


7 


0.17 


Reading 1999 


5.02 


12.58 


2.68 


-0.56 


10.60 


1.87 


21 


0.04* 


Math 1999 


5.81 


12.81 


2.73 


0.13 


11.49 


2.13 


21 


0.02* 


Language 1999 


4.31 


11.29 


2.41 


-0.69 


9.32 


1.79 


21 


0.04* 



Note. * Indicates significance p= .05 

To determine df for 1998-1999 sample: 

Number of matches N=8 

Total df=N-l,8-l=7 

To determine df for 1999-2000 sample: 

Number of matches N=22 

Total df=N-l,22-l=21 



The data set was also examined to gain information about the role of experience in developing teacher competency. 
For teachers that were in both the 1998-1999 and the 1999-2000 set of data, we had hoped to look at whether teacher 
experience effects student SAT 9 scores, and more particularly, whether the differences in performance between the 
certified teachers and the uncertified teachers was moderated as a function of the increased experience of the 
uncertified teachers. But the sample of teachers for whom we had data across two years was very small (six pairs 
across the two years), so no confident answers to these questions can be offered. 

One of our analyses was a one-way within-subjects ANOVA, with the factor being experience, as measured in time 
from 1998-1999 to 1999-2000. The dependent variable was the student's achievement scores for these teachers as 
measured in Normal Curve Equivalents (NCE) for reading, mathematics and language in 1998-1999 and 1999-2000. 
The results indicate that there is no significant difference in NCE scores from the first year to the second year. 
Nevertheless, the scores for each subtest of the SAT increased from the first year to the second, indicating that teacher 
experience may affect the achievement test scores of their students. The means and standard deviations are provided 
below, in Table 1 1 . 

The scores increased from one to two NCE points in each of the three subtests, with the increase in mathematics being 
the greatest. The difference in the scores between the first year and second year are provided in Table 12. We also 
ran an ANOVA on these changes over time, and those results are given in Table 13. 

Table 11 

Means and Standard Deviations for 



Matched Teachers with Two Years of Data 



SAT 9 Subtest and Year 


Mean 


Std. Deviation 


N 


Reading 1998-1999 


36.50 


14.25 


12 


Reading 1999-2000 


37.79 


7.26 


12 


Math 1998-1999 


39.03 


11.49 


12 


Math 1999-2000 


41.07 


8.38 


12 


Language 1998-1999 


35.85 


13.00 


12 


Language 1999-2000 


37.05 


7.38 


12 



Table 12 

Difference in Mean SAT 9 Scores for 
Matched Teachers with Two Years of Data 



SAT 9 Subtest 
and Year 


Mean of 
Differences 


St. Dev. 


St. Error 
of Mean 


95% Conf. Int. 
Lower Limit 


95% Conf. Int. 
Upper Limit 


t 


df 


Sig. 


Reading 


-1.29 


14.54 


4.20 


-10.52 


7.95 


-0.31 


11 


0.38 


Math 


-2.04 


9.29 


2.68 


-7.95 


3.86 


-0.76 


11 


0.23 


Language 


-1.20 


11.71 


3.38 


-8.64 


6.24 


-0.36 


11 


0.36 



Table 13 

ANOVA for Teachers with Two Years of Data 



SAT 9 Subtest 


F 


df 


Error df 


Sig. 


Reading 


.09 


1 


11 


.77 


Mathematics 


.58 


1 


11 


.46 


Language 


.13 


1 


11 


.73 



In order to evaluate whether the differences between certified teachers and under-certified teachers, with two years of 
data, remained similar, grew or decreased from the first year to the second, mean NCE scores for each group were 
analyzed. Results indicate that the difference between the scores of certified teachers and the scores of under-certified 
teachers for the 1998-1999 to 1999-2000 data set, as measured in NCE scores, decreased in reading and language, but 
increased in mathematics. These results are provided below, in Table 14. 

Table 14 

Difference between Certified and Under-Certified Teachers* 

NCE Scores from 1998-1999 to 1999-2000 for 
Teachers with Two Years of Data 



Academic Year 


Reading 


Mathematics 


Language 


1998-1999 


12.93 


7.80 


9.51 


1999-2000 


3.18 


9.47 


4.02 



Note. All scores favor certified teachers over under-certified teachers. 

Discussion and Conclusion 

Many different values necessarily come into play when making educational policy about the qualifications that are 
needed to become a beginning teacher. So much is riding on the performance of these individuals, trusted with 
educating our nation's young. So many skills are needed to do that job well. Thus, a single empirical study of this 
kind cannot provide answers to complex policy questions about the relative benefits and liabilities of allowing 
certified and under-certified teachers to teach our young. Nevertheless, there is every reason to think that the results 
of this study are generalizable and worth considering when educational policies on these issues are debated. 
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As we understand the national situation it appears not to be very differe^^i^that in Arizona. From New York, 



through Chicago, and on to Los Angeles, teachers in schools that serve the poor are often under-certified, 
inexperienced, and may be teaching out-of-field. Teachers who serve wealthier students overwhelmingly hold regular 
certification, have accumulated considerably more teaching experience, and are less often required to teach out-of- 
field. (Darling-Hammond, 1997a, 1997b; 2001; Ingersoll, 2001; Gitomer, Latham, & Ziomek, 1999; Lankford, Loeb, 
& Wycoff, 2002). 

This study addressed one of these factors — the effectiveness of certification on student achievement. We found what 
might be expected of those who choose to do complex work, namely, that those who trained longer and harder to do 
that work do it better. Common sense and empirical data agree. Despite our lack of understanding of how it is 
accomplished, and despite the extreme variability in the programs of instruction (surely masking both excellent and 
dreadful programs), the present research study supports the assertion that university prepared teachers are of higher 
quality than those prepared without an approved program of preparation (see also Evertson, 1984; Darling-Hammond, 
1997a). 



In this study regularly certified teachers significantly outperformed under-certified teachers with children who are 
most at risk of school failure and school dropout. These already low achieving children, when assigned to the 
classrooms of under-certified teachers made gains that were approximately 2 months less per school year on three 
different subtests of the SAT 9. This is about 20% less academic growth than they would have made had they been 
assigned to a teacher with regular state certification. 

The Rowan et al, (in press) study, cited above, states that the relationship between measures of student growth and 
measures of teacher competency are much stronger than are the relationships found when a single years measure of 
achievement is used as the dependent variable, as in this study. Since the districts we studied had relatively large 
percentages of under-certified teachers the odds of a student getting more than one such teacher during their primary 
grades is high. If the magnitude of the effects on student achievement growth over time were as high as Rowan et al. 
believe, then it is likely that exposure to just two under-certified teachers would result in intractable deficits in 
academic growth in reading, mathematics and language. Although their research methods are hard to follow, Sanders 
and Rivers (1996) reach similar conclusions: the effects of poor quality teachers are cumulative. 

In this era of accountability for schools and students, low test performance can mean the loss of employment for 
teachers and administrators, while for students, such results can lead to retention in grade or denial of a high school 
degree. But there are school systems throughout the nation that make regular use of large numbers of under-certified 
teachers and thus, through their hiring practices, virtually guarantee that their students will achieve relatively low 
levels of performance on norm-referenced standardized tests. Students, teachers and administrators will each be made 
to pay for a policy that assures less then desirable outcomes from the school system. 

This situation raises broad questions of policy, such as, what are the causes of, and who is accountable for, the 
placement of the under-certified teachers in the classrooms of our most challenging students? Who should accept 
responsibility for an educational policy that appears harmful and that clearly handicaps students in the lower social 
classes? Will the school districts that make heavy use of under-certified teachers all violate the new federal 
guidelines, since under-certified teachers seem not to be highly qualified to teach? And if these districts will not be in 
compliance with the new federal regulations because they cannot attract qualified teachers to their classrooms what 
can they do differently to receive funding and change the working conditions so that they can attract and keep 
qualified teachers? 



Policy makers should take the results of this study seriously, perhaps also funding more research of this kind to ensure 
the validity of our findings. But meanwhile, on the basis of our findings, we see evidence of a harmful educational 
policy. We believe that those in authority need to attend to the legal and moral issues that arise from our data. It 
appears that we are systematically providing an inferior education to the children of the poor. They start with 
academic difficulties and then through the policies we adopt we handicap them 20% more per year when we assign 
them to classrooms staffed by under-certified teachers. 



The data we have collected also inform us that there is no difference between the performance of new teachers from 
Teach for America and that of all other under-certified teachers. On all tests, and in both years, the certified teachers 
out-performed the under-certified novice teachers from Teach for America. Our results contradict claims made by 
TFA advocates that the enthusiasm and subject-m.atter knowledge, as well as a general education in a prestigious 
university, prepare these recruits to teach adequately in America’s classrooms. The TFA teachers are no better able to 
teach than any other under-prepared teacher. 



In general, research on Teach for America has been limited and the results are often contradictory (Darling- 
Hammond, 1994; 1997a; 2001; Stevens & Dial, 1993; Schorr, 1993; Kopp, 1994; Raymond, Fletcher, & Luque, 

2001). Our findings do, however, directly contradict those reported by Raymond, Fletcher & Luque (2001). We find 
no evidence to support their claim that TFA teachers produce a positive effect on their students' achievement relative 
to teachers recruited in other ways. In our view, the preponderance of the available literature raises serious concerns 
about the TFA program. Although new TFA teachers are required to take a six-week summer training program before 
their school year begins, and they receive support throughout the school year from TFA personnel, the performance of 
their students is indistinguishable from that of student's taught by other under-certified teachers. More important for 
policy makers is that the level of performance of the students^f the TFA teachers was lower than that of the students 
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taught by equally inexperienced but fully certified teachers. That is the more important finding. 



TFA may be a meaningful way for young college graduates to make some money and take a few years out of the 
ordinary path their careers demand. But they are hurting our young, vulnerable, inner-city students. (We expect that 
TFA teachers are faring no better in rural communities, but our data does not address that population,) Because an 
overwhelmingly high percent of the TFA students also leave the profession after their two years of service, their hard 
earned teaching experience will never be put to use with future generations of students. 

While the TFA program appears to be a failure, it is simply part of the larger pattern of failure that attends to the 
policy of hiring under-certified teachers. The policy of hiring under prepared teachers for the schools that serve 
America’s poor looks like an act of class warfare, a concept that Americans find hard to accept. But states have 
adopted, or allowed policies to continue unchallenged, that prevent poor and rural American children from receiving 
the education they need for citizenship or to compete in the economy of the 21st century. These are policies to be 
ashamed of and abandoned. We hope that the new federal legislation will change things, for if it is taken literally, we 
might eventually have highly qualified teachers for all of our nations’ students to learn from. 

Note 

^ This article is based on the first author's dissertation titled Teacher certification does matter: The effects of 
certification status on student achievement, completed Spring, 2002, in the College of Education, Arizona State 
University. The second author received partial funding for helping with this research from the Rockefeller 
Foundation, to whom we are grateful. The views expressed in this report, however, are the sole responsibility of its 
authors and may not reflect the views of The Rockefeller Foundation or the Arizona Department of Education, 
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Abstract 

School restructuring raises questions about the role of school districts in improving student 
learning. Centralization by state governments and decentralization to individual schools as 
proposed in systemic reform leave districts’ role unsettled. Empirical research on the district role 
in the context of ongoing reform is inadequate. This analysis of combined data from the NAEP 
and the Common Core of Data (CCD) was intended to address the issue. We analyzed 1990, 

1992, and 1996 NAEP 8th grade mathematics national assessment data in combination with CCD 
data of corresponding years to examine the extent to which student achievement was related to 
districts’ control over instructional expenditure, adjusting for relevant key factors at both district 
and student levels. Upon sample modification, we used hierarchical linear modeling (HLM) to 
estimate the relationships of student achievement to two district fiscal policy indictors, current 
expenditure per pupil (CEPP) and districts’ discretionary rates for instructional expenditure 
(DDR). Net of relevant district factors, DDR was found unrelated to districts’ average 8th grade 
math performance. The null effect was consistent in the analysis of the combined NAEP-CCD 
data for 1990, 1992, and 1996. In contrast, CEPP was found related to higher math performance 
in a modest yet fairly consistent way. Future research may be productive to separately study 
individual states and integrate the findings onto the national level. 



The role of the local school district is problematic in the on-going school restructuring that focuses on student learning. 
Centralization by state governments and decentralization to individual schools as proposed in systemic reform leaves 
districts’ role unsettled (Elmore 1993). Research is needed to examine the impact of district policymaking on student 
learning. Such research entails linking standardized achievement measures at the student level to district-level policy 
information comparable across jurisdictions. 

This study explored policy and methodological issues relevant to these concerns. We analyzed NAEP data in 
combination with the Common Core of Data (CCD) to examine the extent to which student achievement was related to 
districts’ control over expenditure, adjusting for relevant key factors at both district and student levels. 
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Research Background 



Education reformers face a dilemma in trying to redefine the local school district role in relation to state government. 
On the one hand, the local district, as an independent political entity for local control of public schools, is supposed to 
buffer the schools against external political influences. On the other hand, as a legal creation of the state, the district is 
expected to work as an instrument of state government and to implement state policies and regulations with minimal 
change. The notion of simultaneous decentralization and centralization gives states and individual schools more 
policymaking authority, but leaves the local districts in an ambiguous position (Hannaway 1992; Clune 1993; Keedy 
1994; Marsh 1997). As the authority and responsibility of the state governments continue to expand in public 
education, districts’ role as a governmental unit seems to be diminishing (e.g., see Walberg & Walberg 1994; Elmore 



State governments are playing an increasingly important role in fiscal control and prescriptive policymaking in a wide 
range of areas that were historically the domains of local district decisionmaking. States are contributing a large and 
growing share to local school revenue, specifying requirements on staffing and instruction (General Accounting Office 
1998). State aid formulas have often incorporated specific mechanisms for district inefficiency control, linking 
spending to performance (Duncombe & Yinger 1998; Marsh 1997). Frequently, state governments threaten to and 
occasionally do take over local schools when persistent failure by local administration is confirmed (e.g., lannaccone & 
Lutz 1994; Guskey 1993). 

The districts’ power is also threatened by decentralization in forms of site-based management and the school as a 
professional community (Marsh 1997; Elmore 1993; Clune 1993; Porter 1994; Newmann & Wahlage 1995). State 
education departments now often select individual schools as administrative units for deregulation and interact directly 
with them to encourage local initiatives (Fuhrman & Elmore 1995). Local schools are expanding their options in 
mobilizing community support, making innovations in instruction and curriculum, and making decisions on spending 
and staffing (Elmore 1993; Hannaway 1993). Research appears to support such restructuring as recent studies focus on 
school-level professional autonomy and its link to authentic instruction and achievement outcomes (e.g., Lee, Smith & 
Croninger 1995; Newmann & Wahlage 1995). One implication is that decentralization is reducing bureaucratic 
influence — including district influence — in local school decisionmaking. 



The 16,000 local districts in the U.S. are, however, more than a historical heritage. National data reports describe local 
districts as solid administrative units with diverse conditions (e.g., Levine & Christenson 1998; Protheroe 1997). The 
question for reformers is how to adjust the system to the changing environment, since the underlying logic of American 
federalism requires multiple jurisdictions that compromise different interests and generate productive tension and 
dependence among jurisdictions (Elmore 1993). Available research suggests that it is possible for local districts to push 
reform further by building up strong local constituencies and developing policy initiatives (Fuhrman & Elmore 1990; 
Admundson 1993). This research, however, has been conducted largely with local or state information sources and has 
rarely used national data. To conduct research at national scale requires nationwide data on both student achievement 
and local fiscal conditions. Synthetic analysis of the National Assessment of Educational Progress (NAEP) and the 
Common Core of Data (CCD), explored in this study, seems an efficient approach to this goal. 



Conceptual Framework 

Our basic assumption is that although state governments are constitutionally responsible for public education, local 
districts have an important role in improving instruction. An approach districts take in playing that role is to adjust 
fiscal policy to increase the proportion of expenditure for instruction above the state average. 

Instructional expenditure as a district policy 



School districts differ in instructional expenditure. A recent study used the Bureau of the Census Annual Survey of 
Local Government data from 1980 to 1994 to calculate multiple indicators of cross-district disparity in instructional 
expenditure (Hussar & Sonnenburg 1999). This study found that while disparity in instructional expenditures across 
districts seemed to decline in many states, disparity measures and the pattern of decline varied substantially. 
Furthermore, in a fairly large number of states, the disparity measures were inconsistent, and, in a small number of 
states, disparity increased over the same period of time. While these disparity measures may imply inequity they also 
reflect local fiscal policy differences because instructional expenditures, relative to current expenditures, are more 
subject to government control (Hussar & Sonnenburg 1999). 



Research is unsettled about the relationship between school expenditure and student achievement (for reviews and 
debates, see Conn 1995; Lockwood 1994; Hanushek 1994, 1996; Hedges & Greenwald 1996; Hough 1993). An 
emerging consensus, however, is that overall funding and per-pupil expenditure may be overly simplistic as a predictor 
of learning outcomes, since schooling is conditioned by complicated factors of administration, family, and community 
(e.g., Wainer 1993; Hough 1993). While recent research has provided some evidence supporting the view that 
increased funding relates to better performance (e.g., Payne & Biddle 1999; Wenglinsky 1997), whether overall 
revenue or average per-pupil expenditure substantially affects academic outcomes remains controversial (e.g., 
Hanushek 1996; Hedges & Greenwald 1996). Research needs to relate performance outcomes to variations in resource 
allocation in general and teaching resources devoted to specific program areas in particular (Brent, Roellke & Monk 
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1997; Monk & Hussain 2000). How the money is spent may be crucial to improving achievement (Childs & Shakeshaft 
1986; Conn 1995). Research examining the impact of public school spending, particularly spending for disadvantaged 
schools, suggests that learning can be improved if investment targets improving teacher quality (e.g., Kazal-Thresher 
1993; Ferguson 1991), core curriculum, and standardized achievement (e.g., Lockwood 1994). 

Policy analyses in systemic reform highlight the importance of fiscal control as an incentive/sanction mechanism 
directly linking to academic performance (Lockwood 1994; Elmore 1994). Under this mechanism, schools and teachers 
are encouraged to develop pedagogical initiatives for reaching high-standard curricular goals and are held accountable 
for improving student performance. In this line of thinking, the amount of money spent directly on improving 
instruction/leaming relative to the amount spent on administrative operations and support services should be a crucial 
determinant of achievement outcomes (Childs & Shakeshaft 1986; Elmore 1993). On the other hand, current 
expenditure per pupil (CEPP), which covers spending on broad and immediate local needs, to some extent indicates 
local initiatives and priorities in fiscal control. Encompassing obligatory capital outlay and district discretionary 
spending, CEPP may also be a valid predictor of student achievement. 

In this study, we ask how districts change state prescribed patterns of expenditure and how such changes relate to 
academic achievement. The departure of a district's proportional spending on instruction from the state baseline 
proportion may imply local autonomy in fiscal policy, which has been advocated as a fundamental mechanism for 
maintaining local stake and accountability (Murphy 1994; Strang 1987). Conceptualizing fiscal control as an essential 
element of local autonomy, we attempted to pinpoint a key issue in redefining the district role. We hypothesized that, 
other conditions being equal, a higher district instructional spending rate relative to the state average is associated with 
higher district average math achievement. In addition, we also examined CEPP, a more generic indicator of local 
financial control, by assessing its relationship to student performance. 

Factors related to district spending 

Communities are often constrained from spending more on instruction by local conditions. For example, poor 
communities have to spend more on basic social services, and sparsely populated areas must pay more for 
transportation. A large state contribution to district revenue probably constrains local discretion. The larger the share of 
the state contribution, the more likely the recipient districts fund instruction at a level close to the state baseline. Local 
cost of living is another factor that needs to be considered when examining school resources as it obviously affects the 
spending for teacher hiring and instruction. 



The power of districts varies depending on local socioeconomic conditions, enrollment size, and geographic locale. 
Local socioeconomic conditions as a determinant of local autonomy may alter the intended consequence of district 
policy. Wealthy districts with their well educated populations and organized political support may push reform farther 
and benefit more from focusing on instruction and learning (Elmore 1993). In contrast, poor districts with populations 
of low education and income but high mobility often cannot come up with powerful political support for reform. 
Increasing spending on instruction may compromise desperately needed social services and ultimately undermine 
student learning. The influence of district policy probably also varies across urban-rural areas. Traditionally, rural 
residents are demographically homogeneous within communities but diverse across communities (Nachtigal 1982), 
have strong ties to local schools, and are skeptical about external governance (DeYoung 1990). Within different 
sociodemographic contexts, districts' investment in instruction may result in quite different achievement outcomes. 



Prior studies have explored school funding in connection to standard achievement by synthetic analyses of data from 
CCD and NAEP (Wenglinsky 1997, 1998) or international standardized tests (Payne & Biddle 1999). In particular, 
Wcnglinsky (1997) aggregated achievement measures to the district level and used both school-level and district-level 
variables to predict achievement. Taking the district as the unit of analysis, single-level analyses of aggregated 
performance show that average performance is positively associated with school resources. While such aggregated 
studies deal with the broad issues linking school resources to performance, they were not meant to and did not address 
specific questions concerning district fiscal policy in relation to individual student achievement. Furthermore, there are 
methodological problems in converting NAPE's multi-level stratified sample design into a single-level district sample 
design. The district sample resulting from attaching aggregated NAEP performance data to CCD district records does 
not necessarily represent the national population of school districts. 

In a second study, Wenglinsky used a hierarchical linear model (HLM) technique to analyze merged 1992 NAEP and 
CCD data on 12^^ grade mathematics (Wenglinsky 1998). Again, this analysis did not focus on district fiscal control. It 
addressed the broad concern of school resources in connection to social distribution of academic achievement. Further, 
the study did not distinguish district and school spending measures in the analysis. It did not deal with the 
methodological difficulty stemming from the data merge, namely, the possible unreliability of the estimates resulting 
from the merged data. (Note 1) Our study was designed to continue this line of research by synthesizing national data 
on student achievement and district spending, using sampling adjustment procedures and the technique of two-level 
hierarchical linear modeling. 



Research questions 

The first question in our study was whether or not student achievement varies across districts. The answer to this 



question sets the basis for addressing the substantive concern of district-level effect and for statistical testing of two- 
level models. The second question was how student achievement relates to district instructional spending — and CEPP. 
As the central issue in this analysis, we asked how district mean achievement related to district fiscal policy, adjusting 
for other variables at the district level and sociodemographics at the student level. District fiscal policy was indicated 
by two variables, CEPP and District Discretionary Rate of instructional spending. The latter was simply the difference 
of the district instructional spending rate from the state instructional spending rate. Compatible to the concept 
underlying such disparity measures as the coefficient of variation and the Gini coefficient (see Hussar & Sonnenberg 
1999), DDR should make it straightforward to interpret the HLM estimates of the district fiscal policy effect. 



The third question was: to what extent did DDR and CEPP, together with other district factors, account for the 
achievement outcome variance after adjusting for relevant district-level factors (e.g., enrollment size, state contributed 
share to district revenue, minority rate, poverty rate, geographic locale). In addition, we examined the possible 
interaction effects of DDR with three district conditions: the proportion of district revenue that came from the state, the 
average socioeconomic status (SES), and urban-rural locale. This would allow us to ask whether DDR had different 
effects on achievement under different conditions. 



A final question about the achievement gaps: did higher district instructional spending help reduce math achievement 
gaps associated with race and SES? In other words, did increased instruction spending above the state average not only 
work to promote academic excellence, but also equity? 



Data Sources and Methodology 



For this study, we combined data from NAEP 1 990, 1 992, and 1 996 National Comparison Grade 8 Files and the 
Common Core of Data (CCD) in school years 1989 — 90, 1991 — 92, and 1995 — 96. As the most comprehensive and 
reliable national data source on academic achievement, the NAEP math tests in these three years shared a framework 
supported by the National Council of Teachers of Mathematics (Reese, Miller, Mazzeo & Dossey 1997). 



CCD, covering the universe of U.S. public school districts, provides district-level itemized revenue/expenditures on an 
annual basis (National Center for Education Statistics 1995). It also contains a state file that gives spending data at the 
state level, which can be used to compute DDR. CCD offers information for examining district fiscal policies, 
including core expenditures per pupil, current expenditures per pupil, total expenditures per pupil, percent of total 
instruction expenditures, and percent of total salary expenditures, as well as related state spending measures. 
Additionally information on local sociodemographic conditions is available, including extensive 1990 Census data that 
were incorporated into district records. A linkage file is available for linking NAEP national and state assessments files 
with CCD data for the years between 1 990 and 1 998 (Westat 1 998). 

File Merge 



First, we extracted data from the 1990, 1992, and 1996 NAEP National Comparison of math in 8th grade and CCD 
district files for school years 1989—90, 1991—92, and 1995—96. Using district identification code in both the NAEP 
and CCD files, we merged the two datasets for these years. With assistance from Westat and ETS, the three years' data 
were matched reasonably well. Most districts contained adequate numbers of students for two-level analysis (see Table 



Table 1 

Sample size at district and within-district levels: 
Combined NAEP-CCD data (1990, 1992, and 1996, unweighted) 







Mean within-district 








Year 


District sample size 


student sample size 


Standard deviation 


Minimum 


Maximum 


1990 


144 


43.2 


23.2 


l2.0 


157.0 


1992 


177 


45.2 


27.1 


2l.O 


185.0 


1996 


160 


34.9 


18.6 


10.0 


113.0 



^ For the 1990 data one district (LEAID 3701530) had two cases and was excluded from the analysis. 

^ For the 1992 data two districts (with LEAIDs 5304920 and 1713970) had fewer than three cases and were excluded 
from the HLM analysis. 



The resulting files contain NAEP 8th grader records, with affiliated district variables attached. Additionally, district- 
level cross-product terms were constructed to represent the interaction effects of DDR and the child poverty rate, DDR 
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and urban locale, and DDR and the percentage of district revenue from the state (see Tables 2.1 — 2.3 for unweighted 
descriptive statistics at the two levels). 



Table 2-1 

Descriptive statistics at student- and district-levels: 1996 NAEP-CCD data 



Student Level 1 


Variable name 


Variable label 


N 


Mean 


SD 


Minimum 


Maximum 


DSEX 


Student sex 


5,590 


1.50 


0.50 


1.00 


2.00 


MRPCMl 


Plausible value 1 


5,590 


268.97 


36.46 


127.98 


388.50 


MRPCM2 


Plausible value 2 


5,590 


269.17 


36.03 


120.35 


393.46 


MRPCM3 


Plausible value 3 


5,590 


268.87 


36.52 


124.06 


384.47 


MRPCM4 


Plausible value 4 


5,590 


268.89 


36.01 


138.54 


378.93 


MRPCM5 


Plausible value 5 


5,590 


269.14 


35.87 


121.28 


381.77 


MINORITY 


Non-Asian minorities 


5,590 


0.33 


0.47 


0.00 


1.00 


PARHI_ED 


Parent education 


5,590 


0.58 


0.49 


0.00 


1.00 


District level 2 


ASIER 


Average state instruction expenditure rate 


160 


60.99 


6.48 


50.08 


77.85 


DDR 


District Discretionary instruction spending Rate 


160 


-0.24 


6.78 


-24.99 


17.35 


URBAN2 


Urban district 


160 


0.27 


0.43 


0.00 


1.00 


RURAL 


Rural district 


160 


0.21 


0.41 


0.00 


1.00 


ENROLL_K 


District total enrollment in thousand 


160 


41.84 


110.32 


0.14 


1,049.04 


BLACK_P 


Black student rate 


160 


19.00 


24.75 


0.00 


99.00 


P7118TP 


District poverty rate 


160 


17.01 


11.19 


0.00 


58.50 


PC30ETP 


District at-risk student rate 


160 


3.68 


3.40 


0.00 


15.80 


C_STREVP 


District revenue percentage from state 


160 


47.61 


18.96 


1.80 


78.90 


CURPPE_K 


Current per pupil expenditure in $K 


160 


5.36 


1.43 


3.38 


11.27 


LEV2WT 


District weight 


160 


1.00 


1.37 


0.06 


10.29 


DDR_RUL 


Interaction DDR by rural 


160 


-1.11 


4.18 


-24.99 


6.35 


DDR_URB 


Interaction DDR by urban 


160 


0.40 


2.94 


-18.96 


13.44 


DDR_STP 


Interaction DDR by revenue percentage from state 


160 


-27.66 


355.46 


-1404.46 


921.62 


DDR_SES 


Interaction DDR by poverty rate 


160 


-20.10 


153.74 


-753.19 


529.19 


DDR_RSK 


Interaction DDR by at-risk student rate 


160 


-2.16 


35.07 


-189.98 


112.78 



O 
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Table 2-2 

Descriptive statistics of at student- and district-levels: 1992 NAEP-CCD data 



Student level 1 


Variable 

name 


Variable label 


N 


Mean 


SD 


Minin lum 


Maximum 


DSEX 


Student sex 


8,014 


1.48 


0.50 


1.00 


2.00 


MRPCMl 


Plausible value 1 


8,014 


258.94 


37.53 


130.75 


372.33 


MRPCM2 


Plausible value 2 


8,014 


259.07 


37.93 


127.82 


380.17 


MRPCM3 


Plausible value 3 


8,014 


259.29 


37.91 


110.48 


389.45 


MRPCM4 


Plausible value 4 


8,014 


259.33 


37.65 


126.88 


380.38 


MRPCM5 


Plausible value 5 


8,014 


259.30 


37.74 


123.39 


389.55 
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MINORITY 


Non-Asian minorities 


8,014 


0.36 


0.48 


0.00 


1.00 


PARHI_ED 


Parent education 


8,014 


0.54 


0.50 


0.00 


1.00 


District level 2 


ASIER 


Average state instruction expenditure rate 


177 


60.73 


6.69 


50.08 


77.85 


DDR 


District Discretionary instruction spending Rate 


177 


0.31 


5.20 


-13.54 


16.26 


URBAN2 


Urban district 


177 


0.30 


0.45 


0.00 


1.00 


RURAL 


Rural district 


177 


0.15 


0.35 


0.00 


1.00 


ENROLL_K 


District total enrollment in thousand 


177 


43.93 


98.04 


0.17 


962.27 


BLACK_P 


Black student rate 


177 


15.82 


24.58 


0.00 


99.00 


P7118TP 


District poverty rate 


177 


15.94 


11.57 


0.30 


68.70 


PC30ETP 


District at-risk student rate 


177 


4.03 


4.09 


0.00 


23.60 


C_STREVP 


District revenue percentage from state 


177 


42.40 


20.98 


0.00 


84.60 


CURPPE_K 


Current per pupil expenditure in $K 


177 


4.92 


1.34 


2.98 


10.07 


LEV2WT 


District weight 


177 


1.00 


1.27 


0.06 


8.55 


DDR_RUL 


Interaction DDR by rural 


177 


-0.59 


2.40 


-13.54 


6.42 


DDR_URB 


Interaction DDR by urban 


177 


0.38 


2.59 


-11.16 


16.26 


DDR_STP 


Interaction DDR by revenue percentage from 
state 


177 


5.36 


266.05 


-1063.18 


757.29 


DDR_SES 


Interaction DDR by poverty rate 


177 


-3.26 


100.04 


-472.35 


306.73 


DDR_RSK 


Interaction DDR by at-risk student rate 


177 


-0.05 


27.16 


-162.26 


98.59 



Table 2-3 

Descriptive statistics at student- and district-levels: 1990 NAEP-CCD data 



Student level 1 


Variable 

name 


Variable label 


N 


Mean 


SD 


Minimum 


Maximum 


DSEX 


Student sex 


6,213 


1.48 


0.50 


1.00 


2.00 


MRPCMl 


Plausible value 1 


6,213 


256.24 


33.43 


149.28 


370.23 


MRPCM2 


Plausible value 2 


6,213 


256.28 


33.21 


138.77 


375.86 


MRPCM3 


Plausible value 3 


6,213 


256.61 


33.11 


159.70 


367.46 


MRPCM4 


Plausible value 4 


6,213 


256.14 


33.44 


139.74 


377.80 


MRPCM5 


Plausible value 5 


6,213 


256.32 


33.17 


145.25 


352.46 


MINORITY 


Non-Asian minorities 


6,213 


0.32 


0.47 


0.00 


1.00 


PARHI_ED 


Parent education 


6,213 


0.52 


0.50 


0.00 


1.00 


District Level 2 


ASIER 


Average state instruction expenditure rate 


143 


59.61 


5.71 


49.16 


76.82 


DDR 


District Discretionary instruction spending Rate 


143 


0.19 


7.59 


-20.70 


22.36 


URBAN2 


Urban district 


143 


0.26 


0.44 


0.00 


1.00 


RURAL 


Rural district 


143 


0.37 


0.48 


0.00 


1.00 


ENROLL_K 


District total enrollment in thousand 


143 


37.28 


103.39 


0.09 


918.01 


BLACK_P 


Black student rate 


143 


14.92 


20.48 


0.00 


93.00 
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P7118TP 


District poverty rate 


143 


19.55 


12.35 


1.00 


60.40 


PC30ETP 


District at-risk student rate 


143 


4.50 


4.55 


0.00 


25.30 


C_STREVP 


District revenue percentage from state 


143 


48.43 


18.39 


1.60 


82.40 


CURPPE_K 


Current per pupil expenditure in $K 


143 


4.31 


1.15 


2.41 


8.69 


LEV2WT 


District weight 


143 


0.99 


1.71 


0.01 


8.44 


DDR_RUL 


Interaction DDR by rural 


143 


-1.17 


4.42 


-20.70 


9.21 


DDR_URB 


Interaction DDR by urban 


143 


0.85 


3.10 


-7.24 


15.37 


DDR_STP 


Interaction DDR by revenue percentage from 
state 


143 


10.84 


392.55 


-1,340.28 


1,551.92 


DDR_SES 


Interaction DDR by poverty rate 


143 


2.70 


169.59 


-494.14 


673.09 


DDR_RSK 


Interaction DDR by at-risk student rate 


143 


3.01 


40.59 


-142.15 


153.75 



Sample Modification and Overall Weight 



The resulting student subsamples within districts might not have been reliable in presenting the student populations of 
the given districts, because schools, not districts, were a sampling stage in the original NAEP design. Therefore, we 
reweighted and poststratified the data to improve its statistical reliability. The purpose was to shift the sampling stage 
from the school to the district in order to examine differences across groups of students who were hypothetically 
influenced by local districts' instructional. 

The modification of the NAEP school-student sample into a district-student sample entailed reweighting the original 
sample and establishing the formal statistical status of the district-student sample. Within a sampled PSU, the NAEP 
school sample via "post-allocation" induced a district sample that included districts with which the sample schools 
were affiliated. With modification the representativeness of the school sample to PSUs would lead to the 
representativeness of the district sample to PSUs. The procedure is highlighted below; see Appendix I for details. 



Student sample weighting 



We assigned weights to each sample district within a PSU according to the district’s post-inclusion probability, which 
was defined as the inclusion probability of the union of sample schools in the district. The calculation of this 
probability is described in Appendix I. The post-inclusion probabilities were used as a reference scale for assigning the 
district weights. The weight assigned to a sample district was proportionate to the reciprocal of its post-inclusion 
probability. 

The students within a district were weighted by the reciprocal of the student sampling rate within the district. To 
calculate the weight we used the student population size of the sample district at a given grade. The CCD school file 
provided student enrollment size for each grade of the school, which was aggregated to the district level. We made a 
ratio adjustment (Deville & Samdal 1992; Deville, Samdal & Sautory 1993; Little 1993) of the student weights within 
the districts via poststratification according to important geographic/demographic features. We used this procedure to 
improve the representativeness of the student samples within districts to the district student populations. 

The PSU weights remained intact. The obtained student weights were further adjusted at the national level in the same 
way as the poststratification conducted for the 1992 NAEP (see Wallace & Rust 1994, section 5.1.4). The resulting 
student sample preserved the goal of the original NAEP sampling design; namely, the targeted number of students at 
the given grades for the assessment were selected at a uniform probability nationwide (Wallace and Rust 1994, chapter 
2 ). 

District sample weighting 



Multilevel linear modeling requires the use of level-2 unit (district) weights in analysis to assure that the level-2 sample 
represents the specified population (Bryk & Raudenbush 1992). We weighted the district sample to make it represent 
the national district population using the national district population information from CCD. To improve the 
representativeness of the district sample calibration (ratio adjustment), poststratification was carried out along race and 
Census region. These demographic and geographic characteristics were selected after a comparison of the sample and 
population distributions of those characteristics (Deville, Samdal & Sautory 1993; Little 1993) (see Appendix II for the 
modified sampling distribution of race and Census region). 
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Analytical Approach 



Before data analysis, we edited the data, examined missing data patterns, and constructed indicators to represent 
concepts to be analyzed (see Tables 2. 1 — 2.3 for descriptive statistics of the variables at the student- and district 
levels). 

Expenditure measures 



District instructional spending measures and other fiscal variables were made comparable across locations by adjusting 
for local cost of living and inflation. We used the Teacher Cost Index (TCI) for states and districts available from 
NCES for the adjustment (National Center for Education Statistics 1995; also see Fowler & Monk 2001). TCI, an index 
of costs for hiring teachers, was developed through a regression analysis that estimated the effects of multiple factors, 
including the cost of living and quality of life for each state and district. The TCI score for states were centered by the 
national average of 100 (National Center for Education Statistics 1995). 

We multiplied each state's total capital outlay per pupil by the TCI score for the given state and each district's by the 
district's TCI. Instructional expenditure rates at both district and state were calculated by dividing instructional 
expenditure per pupil with the adjusted total capital outlay per pupil. The resulting district rate was centered around the 
given state rate to generate the DDR in instructional expenditure. This TCI adjustment did not change the values of 
DDRs within a state, but it adjusted the difference in DDR for districts in different states. 

Rescaling data 

We recoded student race/ethnicity into a binary variable (White/Asian vs. minorities) and parent's education into two 
binary variables (with some postsecondary education vs. without). All per-pupil total expenditure measures were 
rescaled in thousands of dollars, total expenditure in millions of dollars, and total enrollment in thousands of students. 

In HLM modeling, each of the district-level variables was further centered around the grand mean, whereas each 
student variable was centered around the district mean. 



Missing data 



We examined nonresponse patterns to assure there were no systematic missing data to bias statistics. Approximately 12 
percent of the 1992 NAEP student records and 3 percent of the 1996 records had missing data on one or more district 
variables due to unmatched records. There were no such missing data for 1 990. We flagged the cases that had no 
district fiscal data in each file with a single missing indicator. One indicator was sufficient because missing data 
occurred quite consistently for most of the district fiscal measures. We then imputed data with grand mean on each 
missing variable and examined the missingness in a series of single-level regular regression analyses that included all 
the predictor variables of math achievement (for rationale see Cohen & Huang 2000; Little & Rubin 1986). We found 
that the missing data were largely random as the regression coefficient for the missing flag was not statistically 
significant in the three years. Thus, in the HLM analysis, we simply used the mean-imputed values for the missing data 
without using the missing flag. 



Two-level hierarchical modeling 



After reweighting the samples, we conducted univariate and bivariate analyses to examine data quality. Single-level 
multiple regression analyses were run with conceptually important variables to explore the general pattern of 
relationships between district variables and math achievement. We also examined the data to make sure that no obvious 
anomalies existed. 

The central research question was whether student achievement was related to district discretionary rates in 
instructional spending (DDR). To address this question with data for each specified year, the two-level modeling took 
the math composite plausible values as the independent variables and DDRs as the predictor variables controlling for 
district- and student-level variables. See appendix III for a formal discussion of the HLM procedure. We used software 
package HLM Windows 5.03, which has a component to run the repeated procedure with multiple plausible values and 
to average the estimated coefficients (see Bryk, Raudenbush & Congdon 1996). 

We conducted the two-level analysis in four steps. First, we examine the extent to which achievement varied across 
districts and the proportion of variability attributed to student effects and to district effects. With a one-way random 
effects ANOVA model, we separated the total variance of math achievement into between- and within-di strict 
components. By assessing the intraclass correlation and reliability of district means, we determined that district-level 
variance was substantial and statistically significant for further modeling. 



Second, we estimated the relationships of a district's average math score to DDR and CEPP, adjusting for the effects of 
total enrollment size, minority student rate, poverty rate, state contribution to the district revenue, and geographic 
locale. With a random-interception model, we examined the extent to which district average achievement related to 
DDR and CEPP, controlling for the district-level variables specified above. 



Next, we explored the interaction effects between DDR and district factors that might have potential combined effect 



on achievement. We asked of the DDR effect on achievement differentiated by some district characteristics. For 
example, DDR was probably more influential to achievement in disadvantaged districts; or, DDR might affect 
achievement in districts that relied more on local revenue than on state contributions. We specifically estimated the 
effect of the cross-product terms of DDR and poverty, DDR and rural-urban locale, and DDR and the state share in 
district's revenue. 

Finally, we tested three important student-level variables — sex, SES, and minority status — in a random coefficient 
regression model. We assess the extent to which these three individual effects varied between districts. Given that these 
gaps did vary across districts, we then tested full models that included these student variables. The full model 
accounted for both district average achievement and district achievement gaps. We first determine whether DDR 
related to district average math score, adjusting for both district- and student-level variables. Then, we asked whether 
achievement gaps due to sociodemographic backgrounds related to DDR (or, whether DDR helped reduce math 
achievement gaps). 

Including student variables in the model was also methodologically important. Students were not randomly associated 
with districts; the district-level estimates might be biased if we did not control for student background effects. Second, 
sex, SES, and minority status at student level were established background factors that strongly related to achievement. 
Controlling for these variables could reduce unexplained level- 1 error and thus improve the precision of the estimates 
of district spending rates as well as the power of hypothesis tests (Bryk & Raudenbush 1992). 



Problems and limitations 

It is typically more difficult to model slopes than means with HLM techniques. Prior analyses of the NAEP data have 
reported unreliable estimation with slope equations (for example, Arnold 1995). While concentrated in intercept 
models, we did test slope models with limited number of variables at the two levels. The resulting slope estimates were 
considerably less reliable than those of intercept models. 

Findings 

Data across the three years appeared reasonably compatible in univariate statistics. At the student level, sex (with male 
coded 1 female coded 2) and minority (with the Asian and White coded zero, all the other groups coded 1) distributions 
were quite steady with data of the three years. Parents' educational levels seemed slightly rising. The rate of sample 
students whose parents had at least some college education was 52 percent in 1990, 54 percent in 1992, and 58 percent 
in 1996. The NAEP math achievement on average was higher in 1996 than in the other two years (approximately 268 
in 1996, 259 in 1992, and 256 in 1990) with compatible standard deviations and ranges (see Tables 2-1 to 2-3). 



At the district level, the descriptive statistics were reasonably compatible across years as well. The average state 
instructional spending rate (ASIER) barely differed over the years, ranging from 59.61 to 60.99 percent. DDR averaged 
very close to zero because by definition the scores were centered by the state average. This measure's standard 
deviation was similar across years though the range shifted slightly. CEPP on average rose from $4,3 10 in 1990 to 
$5,360 in 1996. The average proportion of district revenue that came from the state (C_STREVP) also seemed 
acceptable, with estimates ranging from 42.40 percent in 1992 to 48.43 percent in 1990. These sample estimates from 
the NAEP-CCD combined data were quite consistent with the released national statistics (e.g.. National Center for 
Education Statistics 1996). Likewise, district level demographic statistics (total enrollment. Black student rate, poverty 
rate, and at-risk student rate) appeared reasonable for the three years. Tables 2.1 — 2.3 also present descriptive statistics 
for the interaction terms between DDR and district demographic variables. 

A possible anomaly was with the geographic locale. While the rate of urban districts ranged 26 to 30 percent for the 
years— again reasonable estimates — the rate of rural districts shifted somewhat excessively from 15 percent in 1996 to 
37 percent in 1990. This problem could be indicative of the unreliability of the merged NAEP and CCD data. 

Bivariate Statistics 

A number of patterns revealed in the bivariate correlation analysis must be noted. First, fiscal measures at district level 
were correlated, often in large magnitude. The total expenditure and current expenditure, for example, had a correlation 
coefficient around 0.98 in all three years (the full matrix of the correlation coefficients is available upon request to the 
first author). Per pupil spending items (for example, CEPP, per pupil total expenditure, and per pupil instruction 
expenditure) were correlated to each other. These measures were substantially correlated with the total expenditure 
measures, albeit to a moderate extent. Expenditure measures were also closely related to enrollment size. Table 3 
shows the estimates of bivariate correlation among the selected district-level variables. 



Note that even in this selected group, there were pairs of variables that were well correlated. Proportion of district 
revenue from the state was strongly associated with CEPP (0.68 in 1996, 0.70 in 1992, and 0.66 in 1990). Poverty rate 
and at-risk student rate were also strongly related (0.62, 0.75, and 0.40, respectively, for the three years). We excluded 
at-risk student rate from the equation because it presumably overlapped with poverty rate to indicate the disadvantaged 
condition of a district, yet its range was smaller than the poverty rate. For similar reasons, we dropped rural locale and 



retained urban locale to make the model parsimonious. (Note 2) 



The high multicollinearity required model specification with high selectivity of independent variables at the district 
level such that the included variables were not highly correlated to each other and adequately represented our 
conceptual model. After testing OLS regression analyses with SAS and an initial run of two-level models with HLM, 
we decided to specify in the final model the following independent variables at the district level: 



• District discretionary rate in instructional spending, 

• State average instructional spending rate, 

• CEPP (in thousands of dollars), 

• Proportion of revenue from the state, 

• Total enrollment (in thousands of students), 

• Black student enrollment rate, 

• Child poverty rate, and 

• Urban locale (in contrast with suburban). 



Table 3 

Correlation coefficients of the selected district level variables: NEAP-CCD district level data 1996 

1992 and 1990 (weighted with district weight) 



1996 (N=160) 


State average 
instruction 
spending rate 


(1) 


(2) 


(3) 


(4) 


(5) 


(6) 


(7) 


(8) 


DDS (1) 


*-0.19 
















Total enrollment (thousand) (2) 


-0.05 


*0.26 














Black student rate (3) 


*-0.30 


0.03 


0.15 












Current expenditure per pupil in $k (4) 


*0.68 


-0.12 


-0.03 


-0.15 










Poverty rate (5) 


*-0.28 


*-0.22 


0.00 


0.15 


-0.11 








At-risk student rate (6) 


*-0.34 


0.02 


0.05 


*0.28 


-0.14 


*0.62 






Percent of revenue from state (7) 


*-0.29 


*-0.19 


-0.14 


0.07 


*-0.24 


0.12 


0.08 




Urban district (8) 


-0.06 


*0.22 


*0.24 


*0.23 


0.02 


0.02 


*0.18 


-0.10 




Rural district (9) 


0.04 


*-0.50 


-0.16 


*-0.18 


-0.08 


0.10 


-0.16 


*0.29 


*-0.31 


1992 (N=177) 


















DDS(l) 


*0.16 
















Total enrollment (thousand) (2) 


-0.03 


»0.18 














Black student rate (3) 


0.07 


0.08 


-0.10 












Current expenditure per pupil in $k (4) 


*0.70 


-0.01 


-0.06 


-0.02 










Poverty rate (5) 


*-0.30 


*-0.28 


-0.07 


-0.10 


*-0.20 








At-risk student rate (6) 


*-0.31 


-0.13 


-0.10 


-0.12 


*-0.21 


0.75 






Percent of revenue from state (7) 


*-0.27 


*-0.18 


0.06 


-0.14 


*-0.40 


*0.35 


*0.38 




Urban district (8) 


0.00 


0.11 


*0.32 


*-0.19 


-0.04 


*0.16 


0.10 


0.08 




Rural district (9) 


*-0.28 


*-0.47 


-0.11 


-0.02 


*-0.17 


*0.16 


0.09 


0.13 


*-0.22 


1990 (N=143) 


















DDS(l) 


*-0.34 
















Total enrollment (thousand) (2) 


0.00 


0.14 














Black student rate (3) 


*-0.24 


*0.27 


0.14 












Current expenditure per pupil in $k (4) 


*0.66 


*-0.25 


0.03 


*-0.20 










Poverty rate (5) 


*-0.43 


0.03 


0.02 


*0.32 


*-0.29 








At-risk student rate (6) 


*-0.29 


0.11 


0.08 


0.55* 


*-0.30 


*0.40 






Percent of revenue from state (7) 


-0.12 


0.09 


-0.04 


0.12 


-0.05 


*0.31 


0.01 




Urban district (8) 


-0.01 


0.12 


*0.34 


0.14 


0.08 


0.02 


0.08 


-0.13 




Rural district (9) 


*-0.24 


-0.15 


-0.13 


0.05 


*-0.17 


*0.41 


*0.17 


*0.32 


*-0.21 



' p<0.05 
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HLM Modeling Results 



Unconditional models were tested with each year's data. Without any independent variables, the models separately 
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estimated the variance of the student math achievement at individual and district levels. Table 4 presents the estimates 
of the unconditional models. For each year, a substantial proportion of variance occurred at the district level, meaning 
that districts differed in their average math achievement. For 1996, 26 percent of the variance was at the district level, 
as indicated by the intraclass correlation coefficient. A chi-square statistic of 2401 with 159 degrees of freedom was 
highly significant for the district variance estimate. The reliability of school means (0.71) was also acceptable for 
further HLM analysis, as these sampled school means on average represented the true school means reasonably well. 
(Note 3) The two-level variance distribution pattern was similar in the 1992 and 1990 data. These statistics strongly 
suggested that HLM modeling with random effect at the both levels was necessary for analyzing student achievement 
in relation to district fiscal variables. 



Table 4 

Distribution of variance at the student and district levels: Unconditional models with the 1996, 1992, 

and 1990 NAEP-CCD data 



Estimates 


1996 NAEP-CCD 


1992 NAEP-CCD 


1990 NAEP-CCD 


Fixed effect 




Coefficient 


270.06 


260.39 


256.29 


Average district mean yqo 


(1.47) 


(1.66) 


(1.65) 


(standard error) 








Random effect 




District level variance uqj 


245.00 


373.39 


262.38 


df 


159 


176 


143 


chi-square 


2401.18 


2950.85 


2950.49 


p value 


0.000 


0.000 


0.000 


Student level variance component ry 


975.56 


1160.06 


681.59 


Intraclass correlation rho 


0.20 


0.24 


0.28 


Reliability of the district mean Xj 


0.71 


0.78 


0.71 



District average math achievement in relation to DDR 

Using district variables to explain district average math achievement, we tested a series of means-as-outcomes models. 
At each level, the model specified a random effect. Fixed effects, however, were only specified at the district level, 
including state average instructional spending rate, DDR, total enrollment in thousands of students. Black student 
enrollment rate, CEPP in $1000, child poverty rate, at-risk student rate, urban and rural locale (both coded in contrast 
to suburban), and percent of revenue from the state. The resulting estimates are presented in Table 5.1 . 

Entering district-level independent variables in the model helped account for approximately a half of the district-level 
variance (49 percent in 1996, 62 percent in 1992, and 53 percent in 1990). Controlling for other variables, several 
district variables were found related to district average math achievement in one or more years. District total 
enrollment was related to low average math achievement in 1992, but not in 1990 and 1996. The minority enrollment 
rate was strongly related to low average achievement in all three years, with high statistical significance. The child 
poverty rate was related to low achievement mean, with a significant estimate for each year. These findings were 
compatible with prior research in school organization and achievement. 

The effect of DDR was not substantiated with data for the three years. Net of the effects of other district variables, 
DDR was not statistically significantly related to district mean math achievement in any year. The estimate in the three 
years was essentially zero since it was trivial in size and not statistically significant. CEPP, on the other hand, was 
related to high average achievement, although the estimate for 1992 was not statistically significant (see Table 5). 

With the NAEP-CCD combined data for the three years, we failed to find evidence to support our central hypothesis 
that high district instructional spending relative to the state average would increase district average performance level, 
holding other things constant. 
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Table 5 

District average achievement and DDR and other district level variables: Means-as-outcomes model 

(standard error in parentheses) 



Estimates 


1996a NAEP. 
CCD 


1992 NAEP- 
CCD 


1990 NAEP- 
CCD 


Fixed effects 








District mean yqq 


268.53 (1.18) 


259.23 (1.19) 


254.86(1.28) 


State average instructional spending rate yqj 


-0.21 (0.24) 


-0.19(0.28) 


-0.64 (0.31) 


District discretionary rate of instructional spending 
(DDR) Y02 


-0.21 (0.17) 


0.42 (0.26) 


-0.20 (0.16) 


Total enrollment in thousands yo3 


-0.01 (0.02) 


*-0.03 (0.01) 


-0.01 (0.01) 


Minority enrollment rate yq4 


**-0.45 (0.06) 


**-0.54 (0.07) 


**-0.36 (0.09) 


CEPP$1000yo5 


**2.74(1.06) 


1.49(1.13) 


**4.16(1.59) 


Child poverty rate yo7 


*-0.21 (0.11) 


*-0.32(0.14) 


**-0.59 (0.13) 


Urban district yq8 


-0.82 (3.82) 


-1.27 (3.01) 


1.10(3.50) 


State revenue percentage yoio 


-0.03 (0.05) 


*-0.16(0.06) 


-0.09 (0.06) 


Random effects 


125.21 


150.41 


121.57 


District mean uOj 








chi-square 


1303.42 


1288.03 


1420.38 


District level variance explained 


0.49 


0.62 


0.53 


Student level variance 


974.56 


1158.68 


671.91 



*p<0.05 **p<0.01 



^ A missing value flag was included in the model but not presented because its coefficient was not statistically 
significant. 

Joint effects between DDR and other district variables 

Exploring the possible joint effects between fiscal and demographic variables at the district level, we tested a number 
of interaction terms and presented selected estimates from the model (see Table 6). We found no evidence of a joint 
effect between DDR and the state contribution to the district revenue in relation to math achievement. The three years’ 
estimates were all trivial and not statistically significant. In other words, the state funding for local districts and 
districts' autonomy in instructional spending did not jointly influence achievement in some peculiar way as we might 
suspect. 



Table 6 

District average achievement accounted for by DDR and other district-level variables and interaction 
terms: Means-as-outcomes model (standard error in parentheses) 



Estimates 


1996a nAEP- 
CCD 


1992 NAEP- 
CCD 


1990 NAEP- 
CCD 


Fixed effects 




District mean yoo 


268.72(1.22) 


259.17(1.31) 


254.95 (1.17) 


State average instructional spending rate yoi 


-0.26 (0.23) 


-0.15(0.27) 


-0.44 (0.30) 


District discretionary rate of instructional spending 
(DDR) Yo2 


*-1.03 (0.52) 


0.87 (0.65) 


0.41 (0.35) 



Total enrollment in thousand yq 3 


-0.02 (0.01) 


*-0.03 (0.01) 


-0.02 (0.01) 


Minority enrollment rate Y04 


**-0.47 (0.06) 


**-0.53 (0.08) 


**-0.36(0.10) 


CEPP $1000 Y05 


**2.89 (1.05) 


1.57(1.13) 


*3.72(1.58) 


Child poverty rate Y05 


-0.10(0.13) 


*-0.4 1 (0.16) 


**-0.58(0.15) 


State revenue percentage yo7 


-0.02 (0.06) 


*-0.15(0.06) 


-0.09 (0.06) 


Urban district yo8 


-1.64 (3.69) 


-0.11 (3.06) 


-0.27 (2.74) 


Interaction terms 




DDR X State revenue percentage yq 9 


0.01 (0.01) 


0.00 (0.01) 


-0.00 (0.00) 


DDR X Urban yoio 


-0.36(0.51) 


0.25 (0.63) 


**1.47 (0.41) 


DDR X Child poverty yoi 1 


*0.03 (0.01) 


-0.05 (0.03) 


-0.02 (0.01) 


RandomtfftcXs 


125.47 


146.52 


116.91 


District mean uOj 








chi-square 


1247.82 


1306.15 


1284.82 


District level variance explained 


0.49 


0.64 


0.55 


Student level variance 


974.18 


1158.68 


671.91 



*p<0.05 ** p<0.01 

^ A missing value flag was included in the model but not presented because its coefficient was not statistically 
significant. 

The combined effect of DDR and urban locale seemed more complicated. It was substantial and statistically significant 
with the 1990 data (the fixed effect estimate YolO “ ^ P ^ 0.001), but virtually nil with the 1992 and 1996 data. 

The positive 1990 estimate implied that higher spending on instruction in a district (relative to the state average) was 
related to higher average achievement for urban districts and the effect size was fairly large. The finding, were it 
substantiated with multiple years' data, would offer important policy implications. 



Another interaction term was between DDR and the child poverty rate, which also resulted in inconsistent estimates 
across years. Only in 1996 was the estimate meaningful (0.03 at p.< 0.05 level), implying that, among high poverty 
districts, higher instructional spending than the state average was related to slight better average math achievement; but 
among low poverty districts, there was no such relationship. Again, this finding would be potentially important should 
it be confirmed. It was not, however, substantiated with data for 1992 and 1990. In fact, the effect disappeared even 
with the 1996 data in subsequent analysis where additional variables were statistically controlled for (see Table 8.1). 
Though the findings on combined effects between district discretionary instructional spending were highly tentative, 
they posed questions for further studies. 

Note that the DDR estimate, again, was trivial after the interaction terms entered into the equation, except for 1996, 
when the estimate was significant at p < 0.05 level, modest, and negative (- 1.03). This exception, as we see, should not 
change the overall finding that DDR was largely unrelated to 8th graders' math achievement. On the other hand, net of 
the additional interaction effects, CEPP estimate remained largely substantial and significant for 1996 and 1990, 
indicating a positive relationship with math achievement. The estimate was not statistically significant for 1992. 



Math achievement gaps associated with sex, race/ethnicity, and SES 

To examine the relationships of DDR to math achievement gaps associated with social and demographic categories, it 
was necessary first to assess the magnitude of the gaps and their variance; that is, how much the gaps varied across 
districts. We specified a random-coefficient regression model wherein individual student sex, race/ethnicity, and 
parental educational attainment were estimated as both fixed and random effects. The estimates are presented in Table 
7. 



Table 7 

Math achievement differences relating to sex race/ethnicity and parents’ educational attainment: 
Random coefficient regression model (standard error in parentheses) 



3 -^ 



Estimates 


19963 nAEP- 
CCD 


1992 NAEP- 
CCD 


1990 NAEP- 
CCD 


Fixed effects coefficient 




district mean yqq 


269.77(1.46) 


260.20(1.65) 


256.05 (1.62) 


sex difference yjo 


0.01 (1.26) 


0.78(1.26) 


0.57(1.30) 


Race-ethnic difference 720 


**-23.33 (2.15) 


**-24.09 (2.00) 


**-20.16(1.90) 


Parents' educational attainment difference 73 Q 


**16.75 (1.45) 


**15.38 (1.38) 


**17.89(1.31) 


Random effects 


**256.42 


**383.49 


**267.97 


District mean uOj 


(136; 2557.89) 


(153; 3106.93) 


(117; 2853.21) 


(df; chi-square) 








sex difference ulj 


**82.69 


**80.35 


**91.67 


(df; chi-square) 


(136; 212.05) 


(153; 220.76) 


(117; 195.57) 


Race-ethnic difference u2j 


**168.08 


**191.28 


**105.77 


(df; chi-square) 


(136; 202.07) 


(153; 233.29) 


(117; 238.53) 


Parents’ educational attainment difference u3j (df; chi- 
square) 


**69.07 


**44.75 


**68.04 




(136;219.91) 


(153; 210.52) 


(117; 227.77) 


Student level variance 


778.71 


962.97 


530.74 


Correlation among district effects 


too tn t22 


too til t 22 


too til t 22 


District mean tQQ 


1.00 


1.00 


1.00 


Sex difference t\ \ 


- 0.22 


- 0.01 


-0.03 


Race/ethnicity difference t 22 


0.25 -0.14 


-0.17-0.33 


0.02 -0.03 


Parents’ education difference t 33 


0.16-0.18 0.44 


0.21 -0.29 0.23 


0.11 - 0.22 - 0.11 


Reliability of regression coefficient estimates 




District mean 


0.73 


0.79 


0.71 


Sex difference 


0.31 


0.33 


0.34 


Race/ethnic difference 


0.31 


0.32 


0.25 


Parents’ education difference 


0.26 


0.23 


0.28 



*p<0.05 **p<0.01 

3 A missing value flag was included in the model but not presented because its coefficient was not statistically 
significant. 



There was no evidence from each year's data that 8 th grade math achievement differed by sex within districts as the 
fixed effect coefficient was close to zero and was not statistically significant. However, the random variation associated 
with sex was substantial and statistically significant across the three years (e.g., for 1996 ujj = 82.69, df = 136, = 

212.05), suggesting that sex difference in achievement varied considerably across districts around a mean of zero. This 
situation was similar to findings from some earlier studies (e.g., Raudenbush, Kidchanapanish, & Kang 1991). Not 
only the magnitude of the gap, but also the direction of the association, could differ across districts. It is possible to sort 
out factors that were responsible for the variance in future analysis. However, limited by the scope of this study, we did 
not include sex as an achievement gap in the subsequent modeling. 



Race/ethnicity was clearly a strong predictor of math achievement for all three years. The fixed effect estimate 720 was 
large (-23.33 for 1996, -24.09 for 1992, and — 20.16 for 1990) and highly significant (p< 0.001 each year), confirming 




that minority students (other than Asian American) on average tended to achieve low on the NAEP math test within 
districts. The random effect for race/ethnicity was substantial and significant, revealing that districts differed in the 
racial/ethnic gap and that the variance needed further explanation. 

As a socioeconomic status indicator, parents' educational attainment was specified in the model. The fixed effect 
estimate 730 was positive and large for each year (16.75, 15.38, and 17.89 respectively for 1996, 1992, and 1990, all 
significant at p < 0.01 level). The estimate suggested that within districts, on average, students whose parents had 
college or more education performed better on the NAEP math test. The estimate of the random effect was also 
substantial and highly significant over the years, revealing that districts differed in the math achievement gap 
associated with family social background. Obviously, the varying achievement gap relating to parents' education 
merited further modeling. 



Because of the specified student-level variables in the model, student-level variance became considerably small relative 
to the estimates from the prior means-as-outcomes model (778.71, 962.97 and 530.74 in Table 7, compared with the 
974.56, 1 158.68, and 681.08 shown in Table 5, respectively, for the three years). 



The covariance matrixes generated from the model with the three years' data provided information about the correlation 
between district-level residual random effects (Table 7, panel 3). One of the consistent findings for all the years was 
that the district average achievement residual variance positively and moderately related to the residual variance of 
parents' education-associated achievement gap (0.16, 0.21, and 0.1 1). This finding implied that, holding other things 
constant, the higher the district average math test scores, the wider the district-level achievement gap between students 
whose parents had higher education and students whose parents did not. Another consistent estimate was the 
correlation between sex gap and parents' education gap (-0.18, -0.29, and — 0.22 for the three years). These estimates 
suggested that, to some modest extent, the wider a district's sex difference in math achievement, the smaller the 
district's math achievement gap relating to parents' education. The remaining estimates of correlation changed 
substantially across years and it was impossible to interpret them without further analysis. 

A final note for the random coefficient model: For each year, while the reliability of the intercept (district average math 
achievement) was reasonably high, the reliability measures for slopes (differences associated with sex, race/ethnicity, 
and parental education) were fairly low. This finding pointed to a frequently encountered problem in HLM analysis of 
survey data; that is, the difficulty to model the slopes (see Bryk & Raudenbush, 1992, pp. 102 — 103). Additional 
analysis was needed to sort out whether the lower reliability for slope statistics was due to the NAEP data collection 
design, small within-district sample, real differences across districts in the statistics, or simply the error variance in the 
slope estimation. 

Final model 



To control for variables at the student and district levels in examining the relationships of DDR and achievement, we 
integrated the previous modeling into a full HLM model, with both means and slopes as outcomes in the regression 
analysis. The resulting estimates of fixed effects and random effects are presented separately in Tables 8.1 and 8.2. The 
findings were consistent with those discussed earlier. Accounting for district's average math achievement, we did not 
observe any effect by DDR. However, CEPP and a number of district factors did relate to the achievement mean, as 
interpreted below. 



Table 8-1 

District mean, racial/ethnic gap, and S£S gap in achievement accounted for by DDR and other 
district variables: Fixed effects (standard error in parentheses) 



Estimates 


1 996 NEAP-CCDa 1 992 NAEP-CCD 1 990 NAEP-CCD 


District average mean yqq 


268.64 


(1.15) 


259.12 (1.17) 


254.89 


(1.22) 


State average instructional spending rate yqi 


-0.27 


(0.23) 


-0.16 


(0.27) 


-0.44 


(0.30) 


[Total enrollment in thousand yo2 


-0.01 


(0.02) 


-0.03 


(0.02) 


-0.02 


co.oi) 


[Minority enrollment rate yo3 


-0.47 


**(0.06) 


-0.53i| **(0.08)1 


-0.36 


**(0.10), 


CEPP in $K y04 


2.85 


**(1.07) 


1.60 


(1.14) 


3.69 


*(1.56)' 


Child poverty rate yo5 


-0.09 


(0.13) 


-0.40 


*(0.17) 


-0.58 


**(0.15) 


Percent of revenue from state Y 06 


-0.03 


(0.06) 


-0.17 


**(0.06) 


-0.09 


(0.06) 


Urban locale yo7 


-1.97 


(3.61) 


-0.27 


(3.02) 


-0.54 


_ (2.79) 


DDR Yos 


-1.02 


(0.52) 


0.52 (0.64) 


0.38 


_ (0.35) 


DDR*urban yo9 


-0.36 


(0.49) 


0.20| (0.64) 


1.48 


**(0.43) 


DDR*state revenue percentage yqio 


0.01 


i0_^ 


0.01 


(0.01) 


0.00 


(0.01) 






[DDR*poverty rate you | 


1 0.03I 


L (0.02)11 -O.OsI 


1 (0.03)1 


1 

0 

0 


1 (0.02)1 


Race/ethnicity gap y\Q 


-23.64 


**(1.76) 


-24.35 


**(1.68) 


-21.55 


**(1.59) 


Average state instructional spending rate y^ 


0.52 


(0.38) 


0.56 


(0.37) 


-0.24 


(0.55) 


Total enrollment in thousands y \2 


0.00 


(0.01) 


-0.01 


(0^ 


-0.03 


*(0.01) 


Minority enrollment rate yj 3 


-0.13 


(0.10) 


-0.07 (0.11) 


0.01 


(0.15) 


CEPP in $Kyi 4 


0.71 


_ (1.74) 


0.99 


(2.28) 


4.08 


(2.47) 


Child poverty rate yi 5 


0.04 


(0.18) 


0.38 


*(0.17) 


-0.08 


(0.23) 


Urban locale yjg 


0.57 


(3.53) 


-4.08 


(3.91) 


-7.37 


(4.39) 


DDR yi7 


0.13 


(0.40) 


-0.16 


(0.54) 


0.43 


(0.36) 


SES (parent education) gap y 20 


15.95 


**(1.22) 


14.81 


**(1.15) 


17.13 


**(1.12) 


Average state instructional spending rate y 2 i 


-0.24 


(0.25) 


0.00 


(0.21) 


-0.05 


(0.32) 


Total enrollment in thousands y21 


0.01 


(0.01) 


0.00 


(0.02) 


0.01 


_ (0.01) 


Minority enrollment rate y 23 


-0.08 


(0.06) 


-0.05 


(0.05) 


-0.07 


(0.09) 


CEPP in $K y 24 


0.35 


(1.04) 


1.63 


(1.14) 


-0.47 


(1.70) 


Child poverty rate y 2 s 


-0.31 


**(0.12) 


-0.10 


(0.09) 


-0.03 


(0.15) 


Urban locale y 2 o 


-1.18 


(2.76) 


0.46 


(2.60) 


-1.03 


(2.27) 


DDR y27 


-0.42 


(0.25) 


-0.01 


(0.21) 


0.00 


(0.18) 



*p<0.05 **p<0,01 

^ A missing value flag was included in the model but not presented because its coefficient was not statistically 
significant. 



Table 8-2 

District mean, racial-ethnic gap, and SES gap in achievement accounted for by DDR and other 
district variables: Random effects (standard error in parentheses) 





1996a 


1992 


1990 


Estimates 


NAEP-CCD 


NAEP-CCD 


NAEP-CCD 


Random effects 








District mean UQj 








df 


132.18 


151.91 


119.94 


chi-square 


125 


142 


105 


variance explained: 


1325.27 


1341.32 


1259.89 


relative to means-as-outcomes- model 


- 0 . 05 b 


b-0.03 


b-0.02 


relative to random coefficient regression model 


0.48 


0.61 


0.55 


Racial/ethnic gap U|j 








chi-square 


152.28 


165.78 


81.67 


variance explained: 


187.61 


212.61 


209.58 


relative to random coefficient regression model 


0.10 


0.14 


0.23 












SES gap ulj 








chi-square 


60.80 


42.14 


64.82 


variance explained: 


204.70 


199.03 


206.78 


relative to random coefficient regression model 


0.13 


0.05 


0.06 


Student level variance 


804.03 


988.08 


545.27 



* p < 0.05 ** p< 0.01 

^ A missing value flag was included in the model but not presented because its coefficient was not statistically 
significant. 

^ The estimate implies that the final model accounted for a smaller portion of the variance of this variable. 

Fixed effects: The fixed effect estimate for DDR in 1996 was yog = -1 .02, not statistically significant (p = 0.051 , see 
Table 8.1). The other years' estimates were close to zero and statistically insignificant as well. This finding, again, 
showed no evidence that district discretionary instructional spending relative to the state average spending would lead 
to higher average math achievement. 



The minority enrollment rate was a consistently strong negative predictor of districts' mean math achievement, with 
estimates of -0.47, -0.53, and -0.36 for the three years, all highly significant. CEPP was quite consistently estimated as 
a positive predictor of district achievement level, with estimates of 2.85 in 1996, 1.60 in 1992, and 3.69 in 1990, and 
statistically significant in 1996 and 1990. The child poverty rate was another predictor with fairly consistent estimates 
over the years (-0.09 for 1996, -0.40 for 1992, and -0.58 for 1990), and the statistic was significant in 1992 and 1990. 
Note that the state's share of revenue for the district yielded a statistically significant estimate only for 1992 (-0.17, p < 
0.01). This suggested that holding other conditions constant, districts that received a higher portion of state contribution 
tended to achieve somewhat lower in the NAEP test in that year, probably due to the fact that states' contribution was 
typically prioritized for low-achieving local schools. 

Estimates of interaction effects from the full model confirmed those from the previous modeling. For one, the 
interaction between DDR and urban locale was quite strong and statistically significant for 1990 (1 .48, at p < 0.01 
level). DDR seemed to connect to the somewhat higher average math score of urban districts relative to districts in 
suburban or rural areas. The 1996 and 1992 data, however, failed to produce consistent estimates and therefore this 
finding is tentative as well. We note that the estimate for the DDR-poverty interaction effect was no longer found to be 
different from zero for 1996 data with the full model. 

Gaps associated with race/ethnicity and parents' education were substantiated with the full model. Controlling for both 
within and between district variables, both race/ethnicity and parents' education had sizable fixed effect on mean 
achievement across the three years. For race/ethnicity, the estimate was -23.64 in 1996, -24.35 in 1992, and -21 .55 in 
1990, significant at p < 0.01 level. Two variables showed some unstable effect on the gap, but only for one year. One 
was district total enrollment, related to a trivially narrowed gap (-0.03 at p < 0.05 level) for 1 990. The child poverty 
rate, on the other hand, related to a smaller racial gap (0.38 at p < 0.05 level) for 1 992, suggesting that in districts of 
high poverty, minority students' achievement was slightly closer to White and Asian students' than it was in districts of 
low poverty. Again, without across-year consistent estimates, this interpretation requires additional analyses to 
confirm. 



Accounting for the gap associated with SES as indicated by parents' educational attainment, none of the fixed estimates 
were statistically significant, except that of the child poverty rate 

(-0.31 at p < 0.01 level) estimated with 1996 data. This effect may be interpreted in a similar way to that for the 
poverty rate and race/ethnicity gap. It may imply that for students who attended high poverty districts, parents' 
education made a smaller difference in their math achievement than it did for students in other districts. Poverty at the 
district level thus seemed to "dampen" the racial and socioeconomic differences at the student level. We may speculate 
that this finding perhaps hints of narrow ranges of both student background measures and achievement measures in a 
high poverty district. In other words, poverty-related social homogeneity might underlie the reduced achievement gaps. 
Without consistent results across years, such a "dampening" hypothesis about district-level poverty on the individual 
achievement gaps awaits additional analysis to substantiate. 

Random effects: 

The final model's goodness of fit estimates were quite compawl^ for the three years (see Table 8.2). In general, the 

^54 



model has been improved in regard to accounting for the slope but not for the district mean. For example, between- 
district variance as estimated in the final model was actually smaller than it was estimated in the means-as-the-outcome 
model (see Table 6), which contained the same district-level variables as did the final model. This pattern, similar 
across the three years, may suggest that the multiple predictor variables of slopes caused the model to fit less well as 
most added variables were not related to the slopes as expected. Nevertheless, the variance of the two slopes was 
substantially reduced. Relative to the random-coefficient regression model which contained only random effect for the 
slopes, the final estimates showed that the slope equations fit better. 

Conclusions 

Our findings did not support the hypothesis that drove this study, namely, that local districts could improve academic 
performance by increasing instructional spending. Net of district factors known to affect student achievement, school 
district discretionary spending in instruction, defined as the difference between the district instructional spending rate 
and the average instructional spending rate in a state, did not relate to a district’s 8th grade average math performance. 
The null effect was consistent in the analysis of the combined NAEP-CCD data for 1990, 1992, and 1996. On the other 
hand, fairly consistently, the analysis has found that some district characteristics were related to average achievement. 
Specifically, other conditions being equal, a district’s current expenditure per pupil (CEPP) was found related to higher 
math performance in a modest yet consistent way. Demographic attributes at the district level, including the minority 
student enrollment rate and the child poverty rate, were strongly related to lower math achievement, net of other 
effects. 



The finding of the null effect of local districts’ control over instructional spending as indicated by DDR may lead to 
different directions for future research on the district role in reform. First, considering the complex procedures of local 
decisionmaking on educational financing, research needs to deepen the inquiry about districts’ fiscal policymaking, 
especially the changing patterns of instructional spending in connection to other capital outlays and revenue. Variation 
in instructional spending might reflect distinctive school conditions and policy concerns other than academic 
performance. For example, new technologies and equipment often demand a large budget for upgrading and staff 
training; school security is a pressing issue calling for increasing funding in some areas; aging buildings incur 
extremely high costs to renovate. Many continuing or lump sum expenditures may cause instructional spending to 
fluctuate considerably, which may or may not have an immediate impact on academic test results. On the revenue side, 
federal or state funding and related requirements may vary depending on changes in the government and the legislation 
as well as economic conditions in a larger context. Investigators must consider specific motives underlying decisions 
on spending and the broad conditions leading to such motives, together with DDR in order to understand local fiscal 
policymaking in relation to academic performance. 

The finding that CEPP was positively related to math achievement leads to questions about the value of instructional 
spending as a key indicator of district fiscal control in explaining student achievement. Instructional spending is 
probably a concept too narrow to capture the complexities involved in a district’s fiscal decisionmaking. In contrast, 
current spending adjusted to student enrollment could be more predictive of student performance as it appears 
encompassing such issues as diverse district and community context and various needs for funding. Therefore, future 
research may test alternative indicators of district fiscal policy control in connection to student academic performance 
and other outcomes. CEPP definitely should be a useful candidate for such research. 



Furthermore, we recognize that the district has limited instructional spending control as the range of DDR was quite 
narrow (over the years, approximately 20 percent below or above the state average, see Tables 2. 1 — 2.3). Such control 
may have little impact — relative to district policymaking in other respects — on student achievement. Hence, we may 
want to shift research attention to other aspects of district policymaking and operation. For example, research may 
focus on district differences in handling curriculum and instructional standard development; teacher recruitment, 
training, participation in decisionmaking, and accountability; community involvement and support; effectiveness in 
managing technological upgrading, school and class size, facilities maintenance, student services, and other 
administrative roles. 



TTiis analysis did offer some interesting albeit unsettled clues for learning about how district discretionary instructional 
spending might function jointly with other variables to influence achievement. The 1990 data showed that high DDR in 
urban districts was related to substantially higher average math performance, although the relationship was not evident 
in analysis of the 1996 and 1992 data. On the other hand, with the 1996 data, higher DDR in poor districts appeared to 
relate to slightly higher math achievement than it did in other districts. Limited reliability of the reweighted NAEP- 
CCD data is a likely culprit for the inconsistence. But unknown or shifting circumstances wherein instructional 
spending interacted with local conditions may also have contributed to such inconsistent patterns. It may be promising 
to expand the research to conceptualize and examine such joint effects between local instructional spending and local 
conditions. It is possible that, with more reliable data and precise measures, future inquiries could identify this sort of 
"equalizing” effect on district instructional spending. 

Another intriguing yet uncertain issue that emerged in the analysis was sex difference in math achievement. Within 
districts, sex difference in 8th grade math achievement was estimated to be virtually zero with the three years’ data. But 
between districts, the variance of sex effect was estimated high and statistically significant for all of the three years. 
Factors at the institution level — including school and district — should be part of the explanation. However, classroom 
instructional practice and other organizational environmental influences are probably more salient relative to fiscal 
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policy in explaining the between-district variance of sex difference. Again, this question awaits future investigation. 



Combining NAEP and CCD data into a synthetic analysis proved to be a challenging task. A series of difficulties 
emerged in the process of data merging and data editing because of changes over the years in both data collections (for 
a detailed discussion, see the Technical Report). In addition to addressing technical problems, future research would 
call for re-conceptualizing district fiscal management processes, examining district-state interaction regarding 
education financing, pinpointing joint effects of fiscal policymaking and local/regional conditions, and improving the 
reliability of the measurement and data quality of district fiscal status. 



Specifically, it may be fruitful to separately study individual states and to integrate the findings onto the national level. 
This approach would require using data from NAEP State Tests and CCD for individual state analyses in a much more 
extensive scale than that attempted in the current analysis. It would also entail thoughtful interpretation and 
reconciliation of the possibly incongruent findings from the state analyses to generate cogent narratives about 
education financing at the state and district levels. Such narratives would probably involve in-depth qualitative 
analyses with information sources other than the survey data. A study of this sort would be a large-scale, challenging 
undertaking. But it appears more likely to produce comprehensive and in-depth knowledge about the local district's role 
in performance-driven reform, particularly its fiscal control in connection to student achievement. 
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Notes 

1. In fact, the two studies came up with quite inconsistent results regarding the relationship between central 
administration per pupil expenditure and math achievement. The aggregated study indicated a positive strong 
relationship between the two variables (Wenglinsky 1997 pp. 230 — 231). The HLM study found no relation at all 
between the two measures; rather it found per pupil instructional expenditure to be associated with a smaller 
socioeconomic status (SES)-related achievement gap (Wenglinsky 1998 p. 276). Such inconclusive results call for 
further research to clarify both the methodological and the substantive issues involved in synthetic analysis of NAEP 
and CCD data. 

2. However, recognizing that rural conditions differ from urban districts, we plan to specifically examine the rural- 
urban difference in district spending patterns and achievement during another project. 

3. With the original NAEP data, the reliability for school means is substantially higher, around 0.90 for the three years. 
The lower reliability was apparently a result of sample modification. 
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Appendix I 

Modification of the Sample and Weights 

Appendix I is available for downloading as a Rich Text Fonnatted file. 

Appendix II 

Race-Census Region with Modified Sample 



NAEP-CCD 1990 

Frequencies of students by region-race using sum of Level- 1 weight 10 

Based on original weight (QRIGWT) in NAEP data 



Race-Region 


Freq. 


% 


Cum. Freq. 


Cu. % 


White-Northeast 


877022.5 


12.37 


877022.5 


12.37 


White-Midwest 


1501565 


21.18 


2378587 


33.55 
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White-West 


600752.9 


8.47 


4955076 


69.89 


Non-hispanic black 


1147326 


16.18 


6102402 


86.08 


Hispanic 


771106 


10,88 


6873508 


96,95 


Non-hispanic other 


216087,6 


3.05 


7089596 


100,00 



Frequencies of students by region-race using sum of Level- 1 weight 1 1 

Based on adjusted weight (ADJWT) for merged NAEP-CCD data 



Race-Region 


Freq. 


% 


Cum, Freq. 


Cu, % 


White-Northeast 


959336,4 


13,53 


959336,4 


13,53 


White-Midwest 


1393679 


19,66 


2353015 


33.19 


White-South 


1501023 


21,17 


3854038 


54,36 


White-West 


964005,8 


13,60 


4818044 


67,96 


Non-hispanic black 


1137725 


16.05 


5955769 


84,01 


Hispanic 


838602.6 


11.83 


6794371 


95,84 


Non-hispanic other 


295224.5 


4.16 


7089596 


100,00 



NAEP-CCD 1992 

Frequencies of students by region-race using sum of Level- 1 weight 19 

Based on original weight (ORIGWT) in NAEP data 



Race-Region 


Freq, 


% 


Cum. Freq. 


Cu. % 


White-Northeast 


454207,7 


11.27 


454207,7 


11.27 


White-Midwest 


766619.7 


19.01 


1220827 


30,28 


White-South 


982156,5 


24,36 


2202984 


54.64 


White-West 


523640,3 


12,99 


2726624 


67,63 


Non-hispanic black 


711623.6 


17,65 


3438248 


85.28 


Hispanic 


432971,2 


10,74 


3871219 


96,02 


Non-hispanic other 


160653 


3.98 


4031872 


100,00 



Frequencies of students by region-race using sum of Level- 1 weight 20 



Based on adjusted weight (ADJWT) for merged NAEP>CCD data 



Race-Region 


Freq, 


% 


Cum. Freq. 


Cu, % 


White-Northeast 


446802.1 


11,15 


446802.1 


11,15 


White-Midwest 


753990,9 


18.81 


1200793 


29,96 


White-South 


965832,3 


24.10 


2166625 


54,05 


White-West 


515042 


12.85 


2681667 


66,90 


Non-hispanic black 


635054.6 


15.84 


3316722 


82.75 


Hispanic 


505862.2 


12.62 


3822584 95.37 




Non-hispanic other 


185603.8 


4.63 


4008188 


100.00 



NAEP-CCD 1996 

Frequencies of students by region-race using sum of Level- 1 weight 1 4 

Based on original weight (ORJGWT) in NAEP data 



Race-Region 


Freq. 


% 


Cum. Freq. 


Cu. % 
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White-Northeast 


402880.7 


12.65 


402880.7 


12.65 


White-Midwest 


615135.1 


19.31 


1018016 


31.95 


White-South 


705724 


22.15 


1723740 


54.11 


White-West 


437100.6 


13.72 


2160840 


67.82 


Non-hispanic black 


477674.1 


14.99 


2638514 


82.82 


Hispanic 


402241.1 


12.63 


3040756 


95.44 


Non-hispanic other 


145152.3 


4.56 


3185908 


100.00 



Frequencies of students by region-race using sum of Level- 1 vv^eight 1 5 



Based on adjusted weight (ADJWT) for merged NAEP-CCD data 



Race-Region 


Freq. 


% 


Cum. Freq. 


Cu. % 


White-Northeast 


389907.8 


12.24 


389907.8 


12.24 


White-Midwest 


598667.2 


18.79 


988575.1 


31.03 


White-South 


684636.9 


21.49 


1673212 


52.52 


White-West 


387357.9 


12.16 


2060570 


64.68 


Non-hispanic black 


538638.3 


16.91 


2599208 


81.58 


Hispanic 


432200.4 


13.57 


3031409 


95.15 


Non-hispanic other 


154499.4 


4.85 




100 



Appendix 111 

Analytic procedures with two-level HLM modeling 

Appendix III is available for downloading as a Rich Text Formatted file. 
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Abstract 

This study identifies themes in the theoretical literature on policy implementation that can then 
be used to develop a research-based framework for the scholar about how qualitative research 
can be used to analyze policy implementation through the investigation of informal and formal 
communication lines. This article draws from existing scholarship to bridge the gap between 
policy studies and qualitative research to explore innovative ways for scholars to expand our 
understanding of policy implementation. The article uses the literature to propose a framework 
that can be used to examine policy implementation. The framework is based on the concepts of 
Orientation, Degree, Resources, Activity, Autonomy, Societal Values, Institutional Values, 
Rationale and Power Relationship. 



Understanding policy implementation is difficult enough when sufficient documentation exists to reassemble events 
into a coherent picture. The problem becomes more complex when informal communication lines have been utilized 
to communicate or transfer information (White, 1990). Informal lines are considered to be the people-to-people 
communications such as conversations and often are labeled the "grapevine." Informal lines of communication are 
often used rather than formal lines for facility. Why write a memorandum when less effort is expended through a 
telephone call? Informal interactions such as telephone calls or direct encounters, however, leave little or no archival 
data for the scholar to reconstruct events. The issue of insufficient documentation is especially problematic with 
electronic mail as messages are routinely deleted after an interval of time. White (1990) uncovered frequent use of 
informal lines of communication consisting of unrecorded and unscheduled face-to-face interactions that paralleled a 
formal communication structure. Parallel systems such as the "chain of command" versus the "open door" have 
created problems of miscommunication and misunderstanding when enacting policy implementation (White, 1990, p. 
14). Formal lines of communication were available but were often not used to transfer information or make 
implementation decisions. 

Informal processes include, but are not limited to, conversations, disposable communications such as electronic mail 
and reliance on unspoken understandings such as tradition. The use of informal bureaucratic processes enables a 
rapid and flexible response to difficult and controversial issues. Informal communication processes enable policy to 
be implemented efficiently and effectively; however, they also eliminate written records of decisions and interactions. 
Informal patterns of communication leave few alternatives for the researcher but to rely on qualitative methods to 
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recover policy implementation (Duemer, 1999). Even when written records exist, qualitatively based findings add 
depth and context to the study in question (Blount, 1992). Such context-focused information is sometimes not 
available from archival sources such as memoranda or minutes of meetings that lack such details for purposes of 
brevity (Duemer, 1999). 



Reliance on informal lines of communication, rather than the formal communication structures established by the 
organization, suggests a need to explore and understand how informal communication channels function and their 
impact on policy implementation. Formal communications are directive, regulatory, and structured means of 
conveying information considered necessary for general audiences (Andrews & Herschel, 1996; Weber, 1947). These 
areas can be studied using documentary evidence generated by the formal communication process. Items such as 
memorandums, handbooks, meeting agendas and minutes are readily available to a scholar. The challenge is 
accessing the informal means of communication — the water cooler talks, the lunchroom chats, conversations in 
workrooms — which frequently focus on the content of formal communication documents describing policy. 



A large body of scholarship exists using qualitative methods as means of emphasizing the human element (Manning, 
1990). Such a human emphasis points toward potential usefulness of qualitative research in reconstructing policy 
implemen^tion; however, there does not exist a direct linkage with policy studies. March and Olsen (1976) inform us 
that organizational scholarship must pay particular attention to the human factors that influence decision-making. 
They indicate that personal values and agendas that are not on the surface evident to an investigator often influence 
decisions. Personal factors do not fit into a rational decision-making framework where individual compliance is 
expected; however, personal factors can be accounted for by a focus on the human element. The use of the personal 
element provides a better sense of context (Blount, 1992) through preserving the experiences of those who were 
involved in policy implementation (Manning, 1990). 

The purpose of this article is to identify themes in the theoretical literature on policy implementation that can then be 
used to develop a research-based framework for the scholar about how qualitative research can be used to recover 
policy implementation through the investigation of informal and formal communication lines. This article draws from 
existing scholarship to bridge the gap between policy studies and qualitative research to explore innovative ways for 
scholars to expand our understanding of policy implementation. It is not intended to engage in an exhaustive analysis 
and interpretation of policy implementation as it applies qualitative research. Rather, the intent is to explore some of 
the theoretical literature as a means of provoking scholars to think about ways in which organizational theory informs 
qualitative research. An examination of informal lines of communication and their role in policy implementation can 
yield a more comprehensive understanding of how policies are implemented. 

Discerning Policy Mutation 



Implementation is the means by which policy is carried into effect. Implementation can refer to a one-time effort at 
enacting a policy, or a continuous process such as strategic planning. The implementation process may involve many 
different people and levels of hierarchy, any of which change the nature of policy from decision to implementation. In 
any event, implementation involves the process of moving from decision to operation (Williams, 1976, p. 3). 
Understanding efforts to mutate policy during implementation is essential to recognizing how policy may change 
through implementation, from its original form. 

There would be little need to explore policy mutation if individuals behaved in the same predictable sense as chemical 
reactions. Human reactions would be testable according to proscribed and predictable formulas; however, human 
beings do not behave, they act (Sergiovanni, 1984). "Actions differ from behavior in that they are bom of 
preconceptions, assumptions, and motives, and these are embedded with meanings" (Sergiovanni, 1984, p. 106). The 
thoughts, assumptions, and preconceptions are filtered through values, preferences, prejudices, motives, and the like, 
to produce actions. Prediction is further complicated because actions vary for different individuals even if the 
initiating factor remains unchanged (Sergiovanni, 1984). For example, two individuals in identical administrative 
positions may interpret the implementation of a particular policy in different manners due to opposite philosophical 
perspectives. 




Once an individual or policy-making body sets a policy, there is no guarantee that it will be implemented in the same 
way it was originally intended. The difference between institutions and individuals is central to understanding how 
policy can change from development to implementation. Mutation is more likely when policy is developed in a 
climate that regards implementation as merely a technical detail (Pressman, 1984, p. 143). When a governing board 
directs an institution’s officers to implement a new policy, but does not define any operational limitations or 
delimitations, there is no way to know how implementation will occur or in what manner. Under such conditions it is 
inevitable that implementation will be influenced by individual perceptions. 

Mutation can also occur as policy is processed through the levels of an organization’s hierarchy. One way that levels 
of a hierarchy differ is that some are charged with policy development while others are charged with policy 
implementation. School district central office administrators develop policy that is then implemented by campus 
personnel. Policy can be changed or revised by institutional officials from inception to implementation in a manner 
that more closely meets their conception of what is in their or the institution’s best interests (Elster, 1989, p. 157). 
Individuals can surreptitiously undermine a policy or initiative or at least decline to work actively toward its 
implementation even when they claim to support it (Duemer, 1998; Pressman, 1984, p. 135). For example, a residence 
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life administrator may hinder the implementation of a college's desegregation policy by creating an unwelcome 
atmosphere for incoming minority students (Duemer, 1998). 

Some administrative positions enjoy more freedom or autonomy than others through division of labor (Sergiovanni, 
1984, p. 152; Taylor, 1919; Weber, 1947). Division of labor provides for the development of specialization, 
separation of responsibilities, and more importantly to this article, the means of communication used by the people in 
differentiated roles. Autonomy provides individuals with various degrees of freedom to impose their own 
interpretations on the manner in which policy is implemented (Perrow, 1973). Labor division and specialization 
encourage individuals to identify and congregate into smaller units that share similar goals. This separation also 
contributes to the differences in the lines of communication, which are typically exemplified in the differences 
between formal and informal lines of communication. Persons at higher hierarchical levels (another characteristic of 
labor division) have greater access to more formal communications lines while those at lower levels can more easily 
participate in the informal communication channels of the organization. 

The Role of the Individual in Policy Implementation 

The use of qualitative research methods reflects the idea that institutions are composed of individuals, and those 
individuals should be the focal point of inquiry. In order for an institution to accomplish anything, it must rely on 
individuals. Individuals have their own interests and reflect larger societal interests, any of which may conflict with 
those of the institution. Investigations that focus on individuals seek to understand relationships among those inside 
the institution as well as relationships with those outside the institution. The use of qualitative methods is consistent 
with theory that recognizes institutions to be composed of human will and rejects the idea of institutions as a group 
mind or social reality that is above or beyond human control (Greenfield, 1984, p. 152). Understanding the human 
element in policy is a central aspect of qualitative research, as the human element is the basic unit of social life 
(Elster, 1989, p. 13). Such a perspective recognizes institutions as social constructs which serve society by holding it 
together and ensuring social stability (Elster, 1989, p. 13; Feinberg and Soltis, 1992). Institutions are themselves held 
together and maintained by individuals who share, to varying degrees, similar interests or goals. 



Elster (1989) reminds us that in order to understand policy implementation it is essential to understand the actions and 
interactions of individuals. A human-centered focus, versus an institution-centered focus, avoids the pitfall of 
understanding institutions in terms of key leadership positions such as the study of leadership which is limited to a 
very narrow spectrum of all the individuals in an institution (Greenfield, 1984, p. 160). Such a limited focus 
encourages scholars to remove the personal element and focus on the generic administrator devoid of personal 
identity or interests. Leadership and institutional investigations present a delusive image of administrators and do not 
adequately account for the diversity of individuals and their organizational roles. 



Investigations which emphasize the individual element focus attention on individuals' identification with their own 
interests and breakdowns in communication that increase the likelihood of policy mutation (Perrow, 1973). The 
efficiency of bureaucratic organizations is compromised by the interpretations individuals make in policy 
implementation as the result of their own interests (March, 1984, p. 20). The idea that institutions are rational 
bureaucratic organizations where decisions are regulated by a structure of rules and sanctions is rejected by the 
recognition of individual influence. Institutions have been compared to facades that are intentionally designed to 
mislead observers from the reality that within are individuals who behave as they want (Greenfield, 1984, p. 160). 

IndividuaPs Relationship to Policy Implementation 

Investigations that focus on the role of individuals reject the idea that an institution can embody any value, or that any 
one individual can embody the values of an institution. Such individual focused investigations reflect a perspective 
that recognizes the power of individuals to impact policy implementation and establishes a framework where 
competing values are uncovered and examined to develop an understanding of policy implementation. How do people 
negotiate or reinterpret the policy so as to accommodate their own interests? What can be used to discover these 
individualized interpretations of policy? To further understand the human role, we can frame an individual's 
relationship to policy implementation in terms of Orientation, Degree, Resources, Activity, Autonomy, Societal 
Values, Institutional Values, Rationale and Power Relationship. 



• Orientation: One’s position with respect to attitude, judgment, inclination or interest. Was the individual 
supportive, oppositional, or neutral toward the policy in question? Did the person voice his or her stance on 
the policy? 



• Degree: Scale of intensity or amount. To what degree did the individual support or oppose the policy? If one 
opposed the policy in question, to what degree did that person attempt to stop, obstruct, or mutate 
implementation? Did the individual share his or her opposition or support with others in the organization? 
What means of communication did she or he use to do this? To whom did he or she communicate the stance 
on the policy? 



Resources: Action, money, influence, information, expertise, or measure that can be brought to bear to 
influence or use. What resources were available to the individual that could be used to help or hinder 



implementation? What types of resources did the individual expend on this policy? What resources were 
specifically used in communicating the policy? 

• Activity: Specific deed, action, or function; use of force, influence, or process. What communication actions 
did the individual take to support or obstruct policy? How much communication activity did the individual 
expend to support or obstruct policy? With whom did the individual interact during these communication 
activities? 

• Autonomy: Degree of independence; how closely one has to adhere to prescribed guidelines. A high degree 
of support or opposition will not have had much impact on expense of energy and resources if the individual 
had little autonomy to exert influence on policy. What level of autonomy did that person have in his or her 
position? How does the individual's position influence the communication modes available to her or him? 

• Societal Values: Ideals or customs for which people have an affective regard. How did societal values 
influence implementation? To what extend did the individual accept or reject specific societal values that 
influenced implementation? How did the actions or decisions of the individual change the societal climate? 

• Institutional Values: Professional ideals or customs for which members have an affective regard. How did 
institutional values influence implementation? How are the institutional values communicated to the 
individual? To what extend did the individual accept or reject specific institutional values that influenced 
implementation? How did the actions or decisions of the individual change the institutional climate? How did 
the institutional climate change the actions or decisions of the individual? 

• Rationale: Fundamental, underlying reasons to account for something. What explanation does the individual 
provide for his or her orientation toward the policy? Does the individual have superseding interests, loyalties 
or values that conflict with the policy? 

• Power Relationship: Degree of status relative to individual position. What type of communication, both 
informal and formal, occurred between same or different power levels? 

The preceding questions establish a framework that informs us about individual perspectives toward policy and policy 
implementation. These criteria establish a relationship to policy implementation in individual terms and recognize that 
the relationship between the individual and the organization is reciprocal rather than unidirectional. Additionally, 
these questions can be re-worded to include issues of both informal and formal means of communication. This 
framework also takes into account societal and institutional contexts through investigating communication lines that 
influence individuals, and that individuals change institutions through actions, decisions, and participation in both 
informal and formal means of communication. 



The individual emphasis of the preceding framework is consistent with the work of Bess (1988), that recognizes that 
ongoing and unresolvable differences exist among institutional participants. The framework also accounts for 
divisions of labor and labor specialization, which encourage individuals to identify and congregate into smaller units 
that share similar values, attitudes or perspectives. These smaller units, in addition to organizational divisions of 
labor, contribute to the differences in communication. These in turn not only have an impact in how policies are 
interpreted but also in how they are implemented. Individual interests and breakdowns in communication increase the 
likelihood of irrational behavior and conflict (Perrow, 1973, pp. 2-15). Often the irrational actions are more readily 
recognized with the breakdowns in communication that occur. 

The qualitative scholar can investigate informal communication lines to develop an understanding of how policy is 
developed, implemented, and how it changes in the interim. A people-centered focus encourages us to better 
understand the role of individuals throughout the institutional hierarchy in implementing policy and the influence they 
have in determining its final form. Such an approach recognizes that individuals are not machines, and cannot be 
programmed to consistently perform in a mechanistic and rational manner. The result is a multi-dimensional 
understanding of how policy is affected by individuals. We displace the locus of responsibility when we think in 
terms of how institutions implement policy. Such displacement shifts responsibility from individuals to institutions 
and compels us to assign blame or praise on constructs rather than the individuals who make and implement 
decisions. 

An examination of the roles of the individual and communication in an institution must be understood in a bi- 
directional rather than unidirectional framework. A unidirectional communication focus limits access to 
understanding how a policy is implemented, whereas a bi-directional communication framework expands access to 
learning how an individual's values, attitudes, and perspectives — the human factors — impact policy implementation or 
mutation. Recognition of the role of formal and informal communication channels in organizations is critical in 
discerning the process necessary for effective policy implementation. The framework proposed in this article is an 
initial point for connecting qualitative research and organizational theories regarding communication in policy 
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implementation. It is a framework for exploring how the individual shapes policy and how the institution shapes 
policy through the individual. 

This article recognizes educational institutions to be complex social structures with multiple agendas, rather than 
rational-bureaucratic structures that exist in a vacuum (Dellar, 1 994). As a social process that sometimes involves the 
use of informal, rather than formal bureaucratic protocol, policy implementation is an interconnected part of the social 
structure. 

Where there are internally strong political undercurrents there will co-exist important informal communication 
systems (White, 1990). It is in investigating the role of individuals via a qualitative examination of the communication 
channels that exist within an organization that the scholar can begin to reassemble the factors that influenced the 
implementation of policy. 
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Abstract 

This study should have immediate utility for the United States and beyond its borders, School-to- 
work approaches to comprehensive reform are increasingly expected of schools while legislative 
funding for this purpose gets pulled back. This multisite case study launches the first analysis of 
the New Millennium High School (NMHS) model in Florida, This improvement program relies 
upon exemplary leadership for preparing students for postsecondary education and for a career or 
work. Using participants' feedback, the researcher conducted a pilot study of one prototype 
school and then investigated all 10 original NMHS prototypes. These were examined and 
compared with the benefits of and challenges to the improvement effort in mind. In all, 15 school 
leaders were interviewed and 530 school personnel were surveyed for this sample. It was 
revealed that the millennium program is essentially a grassroots endeavor that continues to 
depend on the state for administrative and financial support. One lesson learned is that school 
leadership can function from deep within the organizational tissues of a restructured school. 
Reformers come in many guises, then, some without formal leadership titles and public 
recognition. Another lesson is that other schools should expect to achieve similar positive effects, 
but ongoing support from the public and government is needed for such significant 
developments. The state policy context of this school improvement model is considered along 
with implications for further change. 



Introduction: School Reform Waves 

A significant wave in school reform is gathering momentum from school -to-work (STW) initiatives that have been 
yielding positive results (Blank, 1997, 1999; Cassel, 1998; Lozada, 1999; Maduakolam, 1999; Mathews, 2000). 
Schools throughout the nation could find the perspectives and assessments provided in this paper useful for 
addressing this issue. High schools are being expected to function differently in their vision, purpose, and effects. 
State and national programs have offered incentives for whole-school reform that help ensure the goals of such 
legislation. Notably, the School-to-Work Opportunities Act of 1994 (Public Law 103-239) required that schools 
integrate academic and vocational education to aid the development all students. However, the status of federal law 
outlining expectations for academic-vocational integration is ambiguous at this time (Medrich, Merola, Ramer, & 
White, 2000). While the laws governing STW legislation lose potency with the withdrawal of the federal government 
from education (Scott, Stone, & Dinham, 2001), the trend toward integrated curricular reform continues to grow, with 
parallel movements across the state of Florida, Texas (Note 1), and elsewhere. This paper is concerned with the 
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manifestation of academic-vocational curricular reform at the state and local level. 



This article represents the first academic analysis of Florida’s showcase school improvement model. The New 
Millennium High School (NMHS) movement grew out ofstate legislation that endorses STW as the basis of 
educational reform. The 10 original NMHS schools offer a vision and process of change that needs study (Florida 
Department of Education, http://www.fim.edu/doe). Such secondary schools that function as career pathways offer 
integrated learning and relevant schooling. They also foster the evolution of school-community organizations that 
partner the schools with business and industry (Mullen, in press; Blank, 1997, 1999). 

The NMHS program depends for its success on exemplary leadership and teamwork aimed at preparing students for 
postsecondary education and for a career or work. However, the process of major change was not new to any of the 
participating schools, and decisions on how to approach this change were influenced by various existing key 
stakeholder groups, not Just the building principal. The strategies used for penetrating the bureaucratic infrastructure 
of these schools enabled us to gain insight into the role of the principal within the millennium context. 

Our results primarily derived from the perspectives of the grassroots reformers themselves. These concur with 
Schuttloffel’s (2000) finding: "In the current school reform environment, crediting successful change to the action of a 
building principal may be as misleading as the assignment of failure solely to the same principal" (p. 3). Our lessons 
learned also reinforce Hargreaves’ (1998) observation about effective principals and their role in shaping teacher 
leadership: "One of the most important functions that educational leaders perform is developing their staff . ..The 
ability and desire [of teachers] to exceed expectations springs from discretionary commitment — from teachers being 
prepared to work above and beyond the official call of duty, entirely of their own volition" (p. 315). 



Two points of validation for undertaking this study emerged during the work: One, the school teams did not have the 
time, as many admitted, to document their developments and problems, and this was viewed as a definite obstacle. 
Two, the millennium model exemplifies the kind of nationwide reform expected of many secondary schools, and they 
will find the results of this study useful as they embark on their transition (Florida Senate, 2001). 

Macro View of Florida's Millennium Schools 

The millennium project at the prototype schools was essentially a grassroots activity guided by the state of Florida. 
Spearheaded by faculty, personnel, and administrators, this improvement program is supported by principals and other 
administrative leaders. This model emphasizes interdisciplinary and cross-disciplinary teaching with a focus on work- 
based learning, team-building, and shared engagement of all critical tasks. 



New Millennium High School is a designation granted to a select number of secondary schools that had initiated 
restructuring plans for a new school-wide curriculum prior to state-level funding. In 1998, policymakers established 
that a prototype curriculum system building on vocational education was needed to prepare students for work and 
postsecondary education (Brawer, 1998). The Florida Millennium Project Task Force claims the NMHS school 
curriculum is "as academically rigorous as the traditional college preparatory pathway" (Brawer, p. 5). The 
representatives in our study provided a similar description, with an added emphasis on the need for schools to develop 
"supportive learning communities with cross-curricular threads and tireless commitment." 



Educational and vocational researchers argue that school -to- work initiatives, when combined with academic 
preparation, can promote many gains. Among these are the achievement of postsecondary goals, Job readiness and 
career development, lifelong learning, and economic self-reliance of students (Maduakolam, 1999; Mathews, 2000). 
A major gain is nationwide and global competitiveness in a response to a trend toward the increased competitiveness 
of nations (Scott, et al., 2001). The relevance of schooling — regarding the academic performance and work readiness 
of students — has been a pressing concern for over a century. It is believed that many high schools continue to operate 
as outdated institutions almost exclusively focused on the college-bound population (Lozada, 1999). A possible 
solution to this seemingly entrenched problem is for high schools to integrate academic and vocational education in 
order to become centers of dynamic, relevant learning. 



The NMHS schools were considered to be capacity-building institutions when funded. (Note 2) For this reason, 
respondents did not view the millennium program as a comprehensive reform effort per se but rather as an expression 
of ongoing change. Being designated as a millennium school was valuable because it provided an opportunity for the 
schools to make improvements along an already traveled path. However, the capacity-building strengths and 
contextual supports of these funded schools ranged widely, and so in some instances the changes seemed more 
significant than in others. Six of the 10 NMHS schools were rural (all of these were small except for one) and four, 
urban (two small and two large). 

Although these were all programmatically accelerated schools, their socioeconomic situations varied considerably: 
Higher poverty and fewer resources characterized the small rural schools. The number of students at the schools 
ranged from approximately 1200 to 3000. The racial make-up of the schools also varied, with minority student 
populations ranging anywhere from 7% to 25%. The minorities were mostly African American with a rising 
population of Hispanics. In one school only, the minority population was relatively high, about 65%. English 
Speakers of Other Languages (ESOL) populations ranged widely across the sites from almost negligible to 40%. 
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These populations were characterized as generally high-poverty and low-literacy, both in English and in the student's 
native language. While the numbers of students at the schools reflected the demographics of the surrounding counties, 
this did not always extend to their ethnic representation. Comments to this effect were: "We have fewer black students 
than blacks in our county but don’t know why." The schools, all federally assisted Title I institutions, had a "free and 
reduced lunch" status. Students from the rural schools were living in circumstances that swung widely from 
comfortable to homeless: "We have kids living on their own, often with cousins, and with their families but with other 
families renting the same apartment." 

Changing demographics and migratory populations were reported to have posed a significant challenge for all of the 
schools. The largest urban school in our sample faced a "tremendous demographic shift" reflected in a student 
population representing 40 countries and over 20 languages: "We have the largest Guatemalan/Mayan population in 
the U.S. And we’ve got language facilitators that speak Hmoung, Konjobal, Urdu in addition to Haitian-Creole." 
Ethnic groups at this school include Muslims, Christians, Jews, Hindus, Azerbaijani, Serbs, and Croats. The migrant 
Hispanic population was in flux at all of the millennium schools, but at this particular school the Hispanic population 
has a 40% mobility rate, the highest in the state. Because of its transitory migrant population, one rural county school 
was actually required to provide a second FTE (Full-Time Equivalency) count to the state. 



As these statistics suggest, the NMHS schools all encountered various challenges of student poverty, diversity, and 
mobility. However, some were better positioned than others for capacity-building. Nonetheless, significant work was 
carried out across the sites, a development that could inspire onlookers. 

Multisite Case Methods 



Pilot Study of Prototype School 

This New Millennium High School research was launched with a pilot study of one prototype school. This early study 
provided the groundwork needed for understanding the "millennium" phenomenon at the school level, and for 
producing tentative results. Through this process, five key areas of leadership were identified and further explored in 
the multisite case study: improved communications and collaborative support structures; development of a career 
guidance program as a lighthouse mechanism for change; encouragement of teacher empowerment, investment in 
research and development, and shared governance; curriculum integration for supporting students’ career 
development; and creating school-community systems of accountability and applied learning. 



These areas were turned into open-ended research questions on the interview/survey instrument, accompanied by 
additional probing questions. For the pilot study that was published, the self-report of the principal was elicited and 
compared with the school-generated data and the staffs progress reports. The school’s changes seemed to bring about 
improvement in many critical areas, notably its overall profile with regard to racial diversity, student attendance, 
graduation, and achievement scores. 

Overview of Qualitative Multisite Methods 

All administrators and staff specialists who were contacted at the 10 NMHS schools agreed to participate. The 100% 
response rate was unusual for a study dependent on the good graces or interest of persons unknown to the researcher 
(principal investigator). As a bonus, we were also given access to reformers functioning deep within the 
organizational tissues of the restructured millennium schools. The typical profile of these individuals was a female 
staff member or specialist without a leadership title. 

The cooperation of the school personnel may have resulted, in each case, from several factors. These include: 1) the 
researcher's lack of affiliation with the state funding agency and the non-bureaucratic, evaluative quality of this study; 
2) the school’s awareness of the value of applied/action research in helping informed changes to be made (Jacobs, 
1991); 3) the school’s desire to gain recognition for its goals and successes; 4) the school’s desire to make its 
criticisms of the millennium program heard by the public and the state; and 5) the school’s eagerness to share with 
other schools and to serve, where appropriate, as a template for change. (A caveat was placed on the role of change 
agent: Because programs were under development, results were not completely known.) 

This follow-up study provided perspective on the millennium effort beyond that underway at one school. Through this 
expanded investigation, various kinds of leaders emerged, busy behind the scenes. For example, the vital role of the 
career specialist was revealed, and so the interviewing was adjusted to monitor this individual’s contribution. Also, 
whereas the principal stood out as the catalyst of the pre-millennium and continuing reforms at the pilot school, this 
was untrue for most of the other schools. 

Data collection. Multiple data for each school included not only taped telephone interviews and surveys but also 
relevant curriculum documents.In all, 15 school leaders were interviewed and 530 school personnel were surveyed, 
with 6 to 26 surveys completed by key players from each school. The interviewee pool consisted of one school 
director, six principals, three assistant principals, three career specialists, one instructor, and a technology coordinator. 
Most of these individuals filled additional roles at their sites that included research coordinator, department head. 



teacher mentor, and dean of students. The interviews lasted between 20 and 60 minutes each. 



Procedures. The same questions were used on the survey instrument as for the interviews. This way, the responses 
could be compared, especially since different stakeholder groups were being consulted. The scope of this study 
therefore extended beyond the principal's self-report to that provided by other school leaders. By involving various 
stakeholders, we were able to cross-reference statements (from the surveys and interviews) to see if the views that 
emerged revealed any patterns. As the next section will show, similar themes emerged that suggest a shared view was 
held by the stakeholders across the 10 sites, except where challenges and circumstances varied and idiosyncratic 
views were expressed. 

The principal investigator and research assistant used established data coding methods in order to analyze the data 
(Miles & Huberman, 1994). Interview transcripts were read independently and analyzed in order to compare results. 
Keywords and ideas were identified and salient pattern themes developed from the transcripts and surveys. Matrices 
were designed to display the overall results. Our conversations about the emergent themes were taped and analyzed. 
Support for claims was established through these systematic approaches to interpreting the data; the instrument itself 
had been validated by the pilot study. 



Thematic Description of Results 

As previously indicated, the change initiatives undertaken at the NMHS schools were mostly in gear before the state 
grant was awarded. Despite challenges, all grant recipients were able to enhance existing or planned programs, and to 
develop some capacity for continued developments. 



The thematic description that follows emerged from the questions on the survey (provided in the form of subheads, 
such as "benefits of the improvement effort") combined with the patterns from the data. The results suggest benefits 
of and challenges to the improvement effort in these specific areas: vision guiding the school, work-based programs, 
role, of school leadership, teacher involvement in change, collaborative character and team work, career academy 
reorganization, professional development and staff morale, student achievement, agendas for further change, ways to 
assist schools, points of pride, and messages for the state and nation. 



Stakeholders' voices are represented in this larger depiction of a grassroots movement that was, paradoxically, 
spawned by state policy reform. Teachers' views have yet to be heard in the debate on educational reform, as their 
voices have "often been muted or stifled in the debates about schooling" (Mitchell & Weber, 1999, p. 187). 

Benefits of the Improvement Effort 

The question of benefits elicited reflection from stakeholders on what it means to be a millennium school. Responses 
converged, highlighting key areas. For example, the value of being acknowledged for doing something worthwhile 
and contemporary for one's students, communities, and state was repeatedly mentioned. One individual commented, 
"It's nice being considered one of the top and to be associated with a select group in Florida." Words such as 
"recognition," "prestigious," "honor," and "strategic" were often used in this context. 



Additional responses covered a myriad of benefits. Notably, these referred to the opportunity to collaborate and 
"learn from other people — obviously that’s part of what it's all about"; to mentor by "taking responsibility for sharing 
the results of our work with as many schools as possible"; to treat the student population as an investment in the 
future: "Most of our students return to our county so we have a vested interest in doing a good job"; to stand out from 
other schools and "model something different"; and, importantly, to make significant changes in practice related to 
program and community development: "This program has given us the opportunity for re-examining practice, 
heightening integration, and making better connections across programs and with people internally and externally." 
Several people provided this important and enduring caveat— "Although being designated as a millennium school 
meant it was neat to be different, this feeling wore off when the state grew disinterested." 



Another benefit was that the schools' popularity increased once perceived as relevant to the times: "Students are 
flooding our school, visiting us with their parents — once admitted, our career teams hook them up with their selected 
vocational field." In terms of public relations, the teams across the NMHS sites increased their advertisement of 
programs, presented to community groups, and involved more people from the community on advisory boards. For 
example, one school's advisory list increased from 43 business partners to over 650 after it was awarded the 
millennium grant. A smaller school currently has 45 businesses that participate in its program, after having had no 
such partnership. 

Finally, the benefit of being a millennium school had apparently set in motion a more inclusive approach to learning 
for all student populations: "Regular high school focuses on students who will go to college where many non-college 
students fall through the cracks." This benefit was simultaneously perceived as an obstacle. Many critiqued how 
America has a rigid view of education that favors the college graduate: "If you don't go to college you're simply not 
successful as a human being. And we're fighting that view, which has to come from the faculty." 



Obstacles to the Improvement Effort 



Not surprising, problems of various types were associated with this millennium effort, a challenging reform initiative. 
(Two schools reported not having experienced any significant obstacles.) One issue concerned difficulty with 
soliciting "buy-in” and contributions from senior staff Most of the schools had perceived their older faculty as 
counter-productive to the changes: "We have small groups of teachers whoVe been here forever and don't want to try 
something new.” It was believed that most senior staff did not understand the relevance of career preparation classes 
or the movement away from strictly teaching academics. At one such school, a leader offered that everyone has a 
responsibility for enabling senior staff members to "feel safer in the new environment." On a positive note, a few 
individuals had experienced leadership from the veteran body of teachers: "Seasoned teachers saw some things that 
needed changing and shared their ideas with the rest of us." 

A second problem raised by half of the schools was that the academy model had to be modified due to less-than- 
idealistic conditions. Most respondents raised the issue of discontinued millennium funding in this context and 
questioned whether the academies could be sustained. They foresee that definite changes will need to be made to their 
academy models. However, the strong conviction about the value of this restructuring model seemed to outweigh 
even these concerns: "Our academy model needs to be kept alive, even if what we end up with is a shadow of its 
former self" 

A problem of significant proportion centered on financial support: "My biggest plea is that there be monies out there 
for continuing the good work already undertaken.” The reports from this small rural school indicate that it is "just 
scraping by.” A consensus was that vocational programs are very expensive to develop because they "require separate 
career support and equipment." To complicate matters, the software, tools, and materials used by the schools were 
apparently not packaged in a "teacher-friendly” way. It proved wearisome to be engaged in "a constant battle to stay 
abreast of technology and software, and somehow find the money to update equipment and replace old and 
malfunctioning computers." In contrast with this picture, two large schools claimed financial stability: "We've been 
very lucky; so far we've had the money to do what we really want, and now we have as many computers as students 
here." These schools were, needless to say, anomalies. 

Speculation about the issue of how state funding is handled for schools undergoing reform led the majority of leaders 
to critique the motives of the state: "I think the whole NMHS thing has never been completely 'bought' by the Florida 
legislature, which is why there's no more money for us.” One person ventured that "The word 'millennium' connoted 
some kind of humanism to the fundamentalist people in the legislature who reacted to that.” Another believed that 
with the NMHS model "you're fighting the idea the public and legislature have that you're going to take a pediatrician 
and turn him into a plumber.” Frustration over this issue of withdrawn funds culminated in the fear that "this 
millennium program will just blow away like everything else in education because we have a history of non-funded 
mandates.” 



Other complaints highlighted the overloading of faculty with teaching and planning responsibilities: "Our teachers 
have to keep up with the reforms while teaching six classes each semester while challenging each other's perspectives 
by hammering away during planning times." Also, teachers had to continue satisfying the state's curriculum 
benchmarks so that students would be prepared for the FCAT (Florida Comprehensive Achievement Test). (Note 3) 
At the same time, the millennium program expected the teachers to "think outside the box, which has been difficult." 
Toward this end, the teachers have been "forced to not use lesson plans as one strategy for helping them to look at 
teaching in new ways." 

As a result of probing questions, some of the school leaders identified fear as an element of teacher innovation and 
change: "All of this [expectation for growth] has been a great consternation to the faculty, very scary actually." As 
Hargreaves's (1998) ground-breaking work on the sociology of emotions suggests, the risk of teachers' work is 
probably compounded in reform contexts that are imposed and associated with high-stakes testing. In the case of the 
millennium program, the teachers* emotions would have been shaped more by a feeling of power than powerlessness 
because of the self-initiated direction of the changes; however, the accountability climate was intense, as was the 
external bureaucratic pressure. 



Notably, managerial issues related to the grants had proven cumbersome and frustrating — new mechanisms had to be 
established. As one leader shared: "We had to create lets of forms and figure out how to record the information and 
code it.” More bluntly put, "The kind of bean counting that I had with this grant was just outrageous. The DOE 
[Department of Education] didn't know the structure of what they wanted in the [curriculum] report until 6 months 
into the grant, and then you had to work retroactively to try to piece together the bean they wanted counting.” (Note 4) 
As another example of a management issue, the marketing of the millennium program proved very trying for some of 
the schools. One shared, "People have an ingrained image of technical education that it's for kids who can't go to 
college or for those academically challenged.” Apparently the mind-set of the guidance counselors and teachers had to 
change before anything different could be expected of the parents and the public. 



Even the notion of leading change was associated with stress because of what seemed implicitly owed to the public: 
"It takes a tremendous amount of courage and commitment to be out front as a 'cutting edge' institution while being 
scrutinized." The public's misguided view of the NMHS initiative as a "been there, done that" step backwards was 
also considered challenging: "We struggle with using new terms to describe new situations to avoid hearing, *Oh, that 
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was tried in 1952 and it failed."' A fresh language (e.g., millennium school and career academy) was hence used to 
foster new understanding. 

Vision Guiding the School 

Respondents described vision in a pragmatic way. Vision was equated with the mission of the school and collective 
efforts toward student improvement. Distinctions were thus not made, for the most part, between "vision" and 
"mission." These concepts were apparently viewed as one and the same within the world of practice. When asked 
what vision currently guides their reform work, leaders shared their schools' mission statements: These supported 
educational learning within a safe environment that is responsive to a continually changing, diverse society. 



Connected to our purposes, the schools' vision/mission reflected the goals for improvement within the context of the 
millennium program. For example, the expansion of currently existing programs was highlighted (e.g., "Faculty are 
committed to creating curriculum that has been negotiated with business constituents and is problem-based with real- 
world applications"), as well as the academic readiness and technical preparation of students for work and for college 
(e.g., "Our student is going to graduate with technical knowledge and relevant skills that will let her get a job while 
she goes to college and at the same time be prepared for college"). These goals meant that the schools aspired to 
provide the learner with a solid knowledge base ("having the foundation students need to be secure and successful"), a 
relevant and progressive education ("preparing students to be on the 'cutting-edge' so they will have more options 
available," and lifelong opportunities ("having all doors open to our students and finances to get them where they 
want to go"). 



The mission/vision of these schools encompassed the demands of the broader community and specifically that learner 
not likely headed for postsecondary education. A typical comment to this effect was, "We need to give those students 
options that aren’t going to college so they will be vocational completers and be Job ready." Indeed, participants hinted 
at the need for a strong political agenda that redefines the schooling process itself to accommodate a vocationally- 
oriented focus: "We changed how we educate so that all of our kids, no matter what they plan to do — go on to college, 
the military, or work, are suited for that purpose." This technical pathway is viewed as integral to, not separate from, a 
rigorous academic curriculum for all: "We’re committed to a Tech-Prep pathway or comprehensive secondary 
education program of study that models academic achievement and rigorous real-world curriculum." 

Work-Based Programs 



The millennium schools used the state funding to implement new programs and strengthen existing ones. The focus 
on the academic-vocational connection, involving curricular development across departments, spawned teaming: "The 
grant allowed us to integrate academics with technical curriculum and to have teachers working side-by-side along 
with their students." Other benefits included making improvements in career academies already in place (e.g., "We 
had the monies to expand the capacity of our engineering and manufacturing academy"), and expanding enrichment 
opportunities provided to students (e.g., "We offered shadowing and internship experiences through the academies"). 
With funding available, personnel (e.g., career specialists) were hired in areas of need, faculty were trained in the 
context of the millennium goals, and expensive equipment was purchased. 



The school teams studied relevant information pointing to areas of work in which employees were needed or would 
be before creating career academies and mustering appropriate expertise. Programs were developed across academies 
or departments in major occupational areas (e.g., business technology, engineering technology, health sciences, 
hospitality/tourism, performing arts, criminal justice, and air force). All programs included pertinent business and 
industry certifications. 

Role of School Leadership 

As indicated earlier, the catalysts within the schools that sparked the millennium program were typically persons other 
than the designated school leader. Principals shared, and with apparent pride, how "the initiative for the new 
millennium grant and model was grassroots, from the faculty up." Another's story of leadership relayed how "a couple 
of people, my assistant principal and career specialist, asked, 'How about if we apply for the millennium grant, since 
we're doing all this stuff pretty much anyway? I said, 'What do you need?' They replied, 'Time to be locked up in a 
room for a few days.' 'Go for it,’ I said." 



Change agents, defined here as millennium professionals committed to ongoing reform within the context of the 
school's collective goals, were hard at work behind the scenes. These individuals, mostly women holding fairly 
modest positions within the school's hierarchy, could otherwise go unnoticed. The person or group that wrote the 
proposal for the millennium grant at each school became the engine for driving the work aimed at both furthering and 
sustaining its positive effects. One message? — reformers come in many guises, some without leadership titles, hence 
unassuming and unsuspected. 



The millennium principals were usually liaisons that interfaced with the district school office to provide 
administrative support and to use power where necessary. These school leaders were not micro-managers. One typical 
response was, "Our principal assumed an administrative role regarding district rules, regulations, and procedures, but 
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did not get involved in spearheading the work or in the day-to-day workings of the program." Principals were also 
active to varying degrees during the implementation of the program as enablers. As one typical principal statement 
was, "I’d been a real leader in changing some of the programs when I came here but excellent people spearheaded and 
wrote the grant." The school's survey data concurred with this explanation of the different leadership roles assumed by 
the principal and the other leaders. 

In exceptional cases, principals whose roles had gone beyond sponsorship to proactive leadership seemed to find it 
necessary. All three had essentially responded to challenging or unusual circumstances. In one instance, the principal 
had set the stage for vision-planning with the faculty, although the staff had, even in this case, discovered the grant 
opportunity and acted upon it. Context is crucial, and at this high school traditional structures that separated faculty 
were so entrenched that the principal "decided to be that person who pushes to get the teachers motivated so they can 
see the 'big picture.'" In another instance, a principal who was newly hired had "brought the millennium vision with 
him and introduced it to the faculty." Yet a third leader, the founder of a new school, envisioned using the new 
millennium program to design the institution. In order to accomplish this, this principal worked alone with a guidance 
counselor. She shared how he preferred to "hire those who could be molded to the philosophical orientation of the 
millennium school, unlike those who had taught for years and were not open to changing." 

Alternatively, the norm that was established for the millennium sites involved leadership "from the trenches" where 
"teachers were very active in changing and improving our school." The staff was prepared for their leadership roles 
and aspired to higher professional development standards. Non-faculty leaders described how they had "offered in- 
service after in-service for teachers and also paid stipends." 



Teacher Involvement in Change 



With the millennium grant in hand, a staff team representing the curricular spectrum typically headed each school's 
program. This team was typically composed of an administrative council made up of department heads working 
specifically with teachers, educating them on the needs of industry and business. But not all of the teachers at the 
schools assumed ownership for the work or were invested in the importance of the changes: "The teachers did not all 
understand the concept of the new millennium and what it could do for our school; in fact, a few felt put upon to do 
extra work." Similarly, but from a career specialist's perspective: "Many teachers are leery of change so I encouraged 
them to see how the initiatives would benefit the students in the long run." The expectation for teachers to change may 
have been experienced by some as pressure to conform, as this statement suggests: "We [the career specialist and 
grant coordinator] recruited and interviewed the academic teachers to make sure they went along with the philosophy 
of the millennium high school — our vocational teachers were already invested." 



The climate in which the millennium program was introduced seemed to vary somewhat from one school to the next. 
However, few schools revealed having to prod teachers to participate and most stressed that participation was 
enthusiastic. Thus, the pervasive climate across the sites appears to have been positive. As one principal reported: "I 
have a very participatory type of administration and everyone is on the leadership team here." Several specialists at 
another site shared how it was their "job to help guide the teachers who were already encouraged, wanting to 
participate to prepare their kids for careers." Similarly, at a different school "all staff who were active brought 
something to the table, either technical expertise or enthusiasm." In some cases staff showed a willingness to commit 
once the funding had been secured: "Once we got the grant and learned how to spend the money, the department 
heads and the vocational teachers became much more involved in staff development and inservicing." 

Collaborative Character and Team Work 

The collaborative work of the teachers and staff included both formal and informal elements, with an emphasis on the 
latter. Most schools, excepting the small rural ones, functioned "totally as teams, with everyone part of some team." 
Faculty and staff formed active technology teams, career teams, curriculum integration teams, community advisory 
teams, and more. An example of curriculum integration teaming was the pairing of academic teachers with vocational 
teachers. Administrators were teamed with particular academies. All work related to the millennium program was 
carried out in teams. 

Teaming facilitated curriculum, integration— departments working effectively intra-departmentally and inter- 
departmentally. Respondents explained that the individual disciplines became cross-curricular in design through such 
efforts as small engines teachers and math teachers team teaching, and English teachers creating curricula with 
vocational teachers. English teachers also partnered with local businesses on curricula aimed at, for example, effective 
job correspondence and interviewing skills. In the smallest rural school, the entire faculty team planned during shared 
breaks. 

Career Academy Reorganization 

The career academy model was defined as a "small school within a larger school whereby students and teachers 
develop identification with each other, which is essential." One practitioner's view of academies referred to "the 
layout of what we call 'teaching-learning spaces' for permitting us more individualized student attention." Career 
academies feature the arrangement of each vocational area (e.g., health education) in a designated building/space. 



Some act as magnets for the school. The overall effort involves collaboration across different program areas and early 
concentration on identifying the career interests of students. The career academy structure reorganizes school 
programs to promote relevant and rigorous curriculum using three major elements: small learning communities, a 
college preparatory curriculum with a career theme, and partnerships with the community. The success of career 
academies in improving student performance could help curtail Florida's 40% high school dropout rate. 

Across the sites this consensus emerged: Regardless of what system of organization was being used, the career 
academy model was, for each, a "work-in-progress." Also, the smaller schools had no choice but to develop their own 
variations on this program. Their arrangements resembled a departmental version of the academy model whereby 
students, enrolled in a program such as health education, took the required courses and received industry certification; 
teachers worked across the curriculum to provide integration. In contrast, the larger schools have found the full 
academy model to be essential for survival: "We're just too large with 3000 students, so we're creating smaller 
learning communities just to make educational learning manageable — modernizing in the process is a plus." One such 
school that has managed to create academies has yet to incorporate academics: "We've got academies with courses 
like carpentry without English and math integrated into them, as that's still on the drawing board." 

Finally, the academy model was valued for the opportunity it gave students to make curricular connections and apply 
essential tenets: "Our kids are seeing the interdependence of all subject areas that combine academics with a career 
and real-world focus." Whether schools used the "pure" or modified academy framework, it was agreed that this 
system helped students who had not fared well in basic, conventional programs. On this note, respondents shared: 
"Our educators reinforce essential course concepts through curricular integration and extension into the community"; 
and "The academies facilitate the diversity of ways students learn and teachers teach, other than 'chalk and talk'." 

Professional Development and Staff Morale 

Professional development may have contributed to increased staff morale for the schools, as this powerful statement 
suggests: "Our school-community partnerships have definitely caused us to rethink, rewrite, open our minds, and 
change our old paradigms of learning." Isolation experienced by the teachers appears to have been ameliorated 
through cross-curricular team planning that was nestled within a local advisory council structure. These school- 
community councils provided realistic assessment and ongoing support: "People representing local businesses meet 
with us across all program areas on a monthly basis, and discuss concerns that the teachers have in the classroom and 
with the new curriculum." In addition to improving relationships, this cooperative learning and advising experience 
has given "the community a 'buy-in' now that they know what goes on at our school." 

Another feature of teacher development was the extensive inservice training sponsored by the schools. Principals who 
were unable to describe in any detail the activities undertaken inadvertently reinforced the role of teacher leadership. 
Estimates of the number of participating faculty and non-faculty personnel ranged from over 50% to 1 00%. At least 
one school's budgetary investment in professional development approximated $1 00,000 annually. Teachers across the 
sites were paid a stipend for training in various areas: technology integration, cross-curricular development, the senior 
capstone project, and standardized test scores. Staff attended many state and national conferences, such as High 
Schools that Work and National Educational Technology. Teachers also found it useful to visit other prototype 
locations to learn more about the on-site implementation of well-developed academies. 

These various modes of work apparently proved motivating. They were, for example, thought to increase teaching 
effectiveness through such means as access to technology. New inclusiveness in decision making also appears to have 
contributed to the morale and empowerment of the staff across the schools, despite frustrations (reported earlier). 
Administrators shared, "Teachers now have choices, such as in the areas in which they want to team and how, and we 
don't tell them what to do — they figure it out." Importantly, faculty were heard to concur: "This millennium 
opportunity has been a big boost for us," and "There's empowerment for us in the ways that the teams work and how 
we want them to work." 

In summary, respondents stated that teacher morale had increased via these three avenues: increased professional 
development opportunities, intensified relations with the community, and improved methods for enhancing student 
learning. This last is exemplified in the following: "Many teachers now have computers — the money has had direct 
impact on making life better for their students." 

Student Achievement 

When queried about evidence for claims that the millennium program had been responsible for increased gains in 
student achievement, no direct correlation was provided. However, factors identified as having made a significant 
difference for one school applied to the others. New and improved conditions, such as smaller class sizes and learning 
communities as well as applied work-based learning, were believed to have contributed to higher test scores. Typical 
comments were: "We came up on SAT's [Sanford Achievement Test, Ninth Edition/S AT-9)] last year [2000] 14% 
over the year before. The fact that we almost doubled the number of students taking them and still went up was a clear 
indication that the millennium program yielded major results for us." Another similarly echoed: "Given our large 
ESOL population, the fact that we came up at all in our scores is amazing." Someone else announced, "Our 
standardized test scores in the last 2 years have gone up and are among the highest scores in the county." 




Through the pilot study that was conducted for this broader investigation, the school's test scores were carefully 
tracked in the context of the millennium program. In all areas of testing and overall success (graduation rate, college 
readiness rate, employment statistics) the school had shown a noticeable increase; scores on standardized tests for 
2000-2001 had even risen above the state and national scores (Mullen, in press). However, as another millennium 
school pointed out, "It’s hard to say how much of our improvement is based on the changes we've made [predictor 
variables] or on other things, such as the different groups of kids that come through every year [intervening 
variables]." Because all of the schools dealt with predictor variables (the millennium program and planned changes) 
and intervening variables (forces not anticipated or controlled, such as changing demographics), no direct correlation 
can be made. Although a corresponding link between the millennium reform and student learning cannot be assumed 
from these changes, the trend in this direction certainly seems promising. 



Indicators of student success include the teacher-student relationship and its personal quality: "This program has 
really brought the whole school into a one-to-one relationship with each student; this process changes attitudes 
because you begin to see each other on a human level." Apparently students and teachers alike appreciated the new 
opportunity for closeness that the academies provided. Supporting this picture, the guidance department, functioning 
as a career team, was uniformly perceived as having become an invaluable support to teachers, students, and parents. 



Advisory boards, in their partnership with the schools, were positively assessed. They were believed to have 
supported student achievement in at least four ways: 1) providing input on instructional development, 2) sponsoring 
on-site internships, 3) assessing student progress, and 4) providing industry and local certification for work-based 
programs. As a major effect of this support, students were generally kept on track by the career teams whose work 
was synergistically enabled through the advisory boards. 

Also, the establishment of career centers had assisted student achievement across the sites. The centers considered 
state-of-the art flourished: "We’ve got a career center with 21 new computers hooked up to the Internet — it's used a 
lot." Students were expected to use technology in their presentations and connect their learning to their career goals. 
For some schools, though, the logistics of using insufficient and outdated software was evident: "Imagine trying to 
prepare kids to enter the business world when you have a business lab that runs on Windows 3.1 — we had to build the 
connections outside our doors." 

Finally, deeper and more lasting effects underscored changes in the value of learning throng practical application 
and real-life experience. The curriculum seemed to shift from being a state-imposed albatross to a source of internal 
engagement: "The millennium model made the curriculum come alive for the kids who could finally see the relevance 
of what they’re learning to the real world." 

Agendas for Further Change 

Major agendas for further improvement emphasized areas needing attention and support. For example, all millennium 
representatives aspire to enhance their relationships with the business/industry community. They aim to increase 
opportunities for industry certification and alignment of the curriculum with the real world. Specifics along these lines 
included "adding more business partners," "increasing our exposure in the community by significantly enhancing our 
work-based learning environment," and "eliciting more buy-in from our school-community for distance learning and 
career initiatives." A few mentioned the need to improve the senior capstone project by working more closely with the 
community. 



A school -based area targeted for further change was the somewhat murky notion of cross-curricular integration. 
School leaders called for clarification from the state regarding the expected dynamics of this kind of work. 
Specifically, they emphasized needing "more academic and technical development and integration to enhance the 
relevance of learning"; "an increased overlap of our vocational areas with academics," and "a deepening of the 
meaning of 'integrated curriculum.'" 



ERIC 



Interestingly, several respondents identified the need for greater ownership over the practice of leadership at the 
school level: "Create an atmosphere that is called leadership — it's important to recognize that you can be a custodian 
and provide leadership for those people you work with." As an example of taking ownership, many articulated plans 
for obtaining funding to continue the work accomplished: "Our main focus is on hunting down more grants to make 
sure we have money for a guidance counselor and career specialist"; and, in summary, "Funding is our top concern — 
to enhance and continue programs." 

Ways to Assist the Schools 

More guidance from the state was requested for carrying out the new millennium work. All agreed that this meant a 
focus was needed on "stronger DOE [Department of Education] support, less equivocation, and a concrete plan that 
doesn't vacillate according to political administration." Others called for assistance with the millennium goal of 
connecting with the real world of business and industry. The state was strongly advised to increase its effectiveness 
vis-a-vis the millennium program by developing "a better reporting system," "a streamlined bureaucracy," and "a 
clearer idea of expectations." Someone summed up these critical sentiments: "You've got a system here in Florida of 
encumbrances." 
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Participants thought that the Department of Education probably needed support from the state: "When the DOE does 
these programs Florida needs to acknowledge them." A strong message sent to the state recommended commitment to 
initiatives beyond the short term: "If you want teachers to stand behind innovation, then it has to be known that these 
are programs everyone is going to stand behind — they shouldn’t just disappear." Scott and colleagues (2001) link this 
concern to a larger pattern of erratic school funding that contributes to teacher discontent and even erosion of the 
education profession. 



The respondents asked for more time to work toward the expectations for outstanding performance. Areas specified to 
this effect were professional development, instructional preparation, curriculum integration structures, and shared 
workloads. The need for training in integrative instructional delivery strategies was underscored. Teachers also desire 
time to "visit other schools to find solutions to similar problems." In fact, some argued that "a comprehensive plan for 
professional development" could only continue with monetary support. Smaller classes were also supported for 
enhancing student learning. 



Points of Pride 



School accomplishments were generally expressed as points of pride. Student success received praise in such areas as 
the senior capstone project, the mock interview process, and especially accountability assumed for one’s education: 
"Our students now feel they own their future and are responsible for it." 

Program highlights were also mentioned as noteworthy. These include the infusion of career development from grade 
9 to 12, the use of business/industry certifications, the high quality of vocational programs, the establishment of 
Bright Future Scholarships and others, opportunities for distance learning, articulation of credits with community 
colleges, responsiveness of business partners to sponsoring internships, active student-run school facilities (e.g., 
restaurants and banks), bustling career/technology laboratories and centers, popular health careers programs, 
increased standardized test scores, and the employment success of graduates. These features all evidently resulted 
from the integration of "real life experiences in the curriculum" combined with the increased motivation of students to 
learn. 



Even the millennium vision/misson itself constituted a source of pride: "We provide a learning environment with a 
practical yet visionary focus on integrating technology and education." 

Messages for the State and Nation 

Most notably, the schools called for improved efforts from the state in the areas of communication, recognition, 
support, and especially funding. In the absence of continued funding, schools are forced into creative solutions such 
as downsizing and reorganizing their academies. 



This optimistic message was sent to high schools interested in adopting the career academy approach to student 
learning: "If we could do it, any school can." Other schools were advised to designate individuals as career specialists 
and program implementers, without teaching responsibilities. One individual encouraged that change can proceed 
more modestly: "Just work with what you have — we did not overhaul the whole school." However, the millennium 
schools generally warned that the effort is very demanding: "It's tough trying to be all things to all people, something 
comprehensive models expect." 



A consensus emerged that the millennium model only makes good sense because it prepares students for the next 
level of their lives toward the "ultimate goals of being self-supportive and better prepared as citizens in the world." 
These programs were reported to be successful learning communities that provide a safe place for learning. 
Enthusiasm was clearly voiced: "The millennium school is a great place to send your kids to school." 



The schools want the public to hear about the millennium concept and to learn firsthand what it can look like in 
practice. Even the most "challenged" school had this to say: "We may only be a small rural impoverished county with 
zero growth, but we've been able to do great things at our school — come visit us and see for yourself" 

Reflections 

This article responds to the call of Lieberman, Saxl, and Miles (2000) to describe how schools actually create 
structures for improvement: "There are few precedents, few models, and no guidelines" (p. 348). With this goal in 
mind, this study of Florida’s site-based improvement model builds upon stakeholders’ models of practice. Our results 
combined with the literature suggest that schools operating as NMHS prototypes can significantly and holistically 
improve (Mullen, in press; Drawer, 1998). Research indicates that students excel when schools offer a dual 
curriculum focused on postsecondary education and the career or workforce (e.g., Berryman & Bailey, 1992; Blank, 
1997, 1999). 



As has been described, the new millennium model attempts to make headway with internal and external development 
as well as accountability to constituents. A premium is being placed on educational institutions that can act as a 



280 



catalyst for the nation's schools in developing a comprehensive, integrated system that closes the pervasive gap 
between academics and work. Generally, schools can probably benefit from exposure to such prototypes that show 
how leadership structures and program development can be redirected to focus on relevance and rigor. 

Schools can also improve by decentralizing and hence increasing their capacity for site-based management through 
school-community control (Mullen, in press; Leithwood, 2001), The NMHS movement, while dependent upon local, 
grassroots involvement for its success, is not to be viewed as separate in its mission or operations from the state: The 
state’s role in endorsing the millennium vision as well as related reforms (e.g,. High Schools That Work) was 
paramount. Local and state leadership can serve to enhance and reinforce one another, Fullan's (1999) change model 
idealistically demonstrates in the context of school reform policy that top-down (state-to-school) and bottom-up 
(school-to-state) leadership can provide the necessary momentum for change toward coherence and cohesion. 



Applying Fullan’s model to this study makes one wonder wherein the balance of power lies between schools and the 
state. For those millennium schools that struggle as impoverished rural sites, their very real dependence on the state 
for financial support will determine their outcomes and successes over time. The reality for the NMHS prototypes as a 
whole is that their financial dependence upon the state is combined with the need for endorsement and approved 
guidelines for reform. Reform cannot occur in a vacuum without accountability to the state and public. 



These funded schools could be viewed as a "legislated" or oxymoronic grassroots movement that is, in many ways, 
similar to other public institutions. Take the accountability of test preparation and scores, for example. The continuing 
responsibility for high performance on standardized tests for these over-loaded schools has heightened the tension 
between the decentralizing effort of the school-communities and the centralizing effort of the state. As has been 
increasingly shown, high-stakes testing weakens the position of schools and communities to control their own 
learning process (Caputo-Pearl, 2001 ; Waite, Boone, & McGhee, 2001), To what extent, then, even the millennium 
model can enable a grassroots base of power to endure remains to be seen. 

Schools that are centralized or aligned with governmental mandates probably overly state the leadership role of the 
building principal. But the authority and even activism of teachers in concert with other stakeholders can shift this 
paradigm and even re-make it (Glickman, 1998), To varying degrees, the original millennium schools represent 
shared governance models, even in those exceptional cases where principals "kick-started" the initiatives and where 
not all teachers participated. The professional capacity of the staff — recognized by Sergiovanni (2000) as a critical 
ingredient in any leadership endeavor or school innovation program — appears to have increased through the 
millennium initiative. 

Not to be overlooked, criticisms of the NMHS model from those who had experienced its daily ramifications were 
varied and at times stark. Notably, one undeniable issue concerned conformity — having to mold to the philosophy of 
the New Millennium High School, Those cast as non-cooperative may have been resistant, not simply disinterested — 
a recognized but untapped dimension of this research. As Goodlad (1984) and Mitchell and Weber (1999) have 
pointed out, teachers can subvert or deflect change when policies of curricula are viewed as imposed and even 
harmful, or not beneficial to student learning. 

Policy Implications for Continued Success 

The Florida Senate Bill (2001) would have promoted the cause of vocational-technical education at the high school 
level, but it failed to become legislation in 2001 . We speculate that the policy, which would have legislated whole- 
school reform for all high schools in Florida, was probably considered overly ambitious and risky. The state 
legislature will need to provide dollars to fully implement any such policy. Schools that are "kick-started" in the 
direction of reform tend to be abandoned, as in the case of the millennium schools. 



The question of sustainability for transforming schools is a national issue (Mullen & Graves, 2000), The literature 
forecasts that STW initiatives may erode or even disappear without federal funds (Hettinger, 1998), The investment of 
both capital and people is critical. It is expensive to fund the goals of millennium schools. Some of them have been 
forced to improvise to such an extent that their reforms may become tarnished. The irony is that schools expecting to 
build self-sustaining systems need continuing support as they work toward this goal because of the protracted nature 
of effort involved. Despite the successes of any school, improvements must be perpetuated and refined if they are to 
have impact over the long haul (Mullen & Graves, 2000), 



The gap between the good intentions of policy and the reality of implementation at the school level is illustrated in 
another case study. This shows how a school’s reliance on resources could lead to an "inevitable social construction of 
social failure" (Schuttloffel, 2000, p, 10). The assumption that schools that have been successfully initiated into 
change through temporary funding can somehow sustain changes on their own is probably short-sighted in most 
cases. 

Although it must be taken into account that reformed schools can be quite resilient, the small rural millennium 
prototypes in particular will probably experience great loss and compromise. As Schuttloffel (2000) aptly sums up, 
school reform legislation must consider the situation of distressed schools if comprehensive reform is to develop 
along the lines envisioned — otherwise the nation could end up being littered with a series of quasi-modified sites at 

281 



best. Support is needed for continuous adaptation within our schools; even automobile industries like Honda of 
America are ahead of schools in these respects (Weiss & Cambone, 2000). Sustained staff planning, shared decision- 
making, and professional development will require a system of supports beyond those available even within the 
millennium sites. 

Possible recommendations from another study of site-based school reform (Weiss & Cambone, 2000) are relevant 
here: Ongoing stipends could be paid for teachers’ extra work hours, continuing professional development, and time 
spent planning the (integrated) curriculum; also, school districts and legislators will have a key role to play in 
supporting the (millennium) reform program over time. Teachers in the NMHS schools currently see that support has 
faded, perhaps when it is most needed. Although they can write grants to continue the work, there are many 
dimensions beyond the financial — structural, informational, and developmental — that need support in order to allow 
visions to take root. 

Finally, this improvement model has implications for change on a broader scale. High schools can modernize and 
adolescents can thrive where a dual purpose of schooling exists. The people we heard from seem to want the 
opportunity to both lead and serve in this respect. And they appear to have gained this opportunity through the 
millennium model, at least in part. In today’s world it might be that site-based reforms within the public secondary 
school system will need to be paradoxically "married” to the state beyond their infancy. 



1. High Schools That Work, Texas Education Agency, http://tea.state.tx.us. 

2. These ’’lighthouse” schools, as we refer to them, had been at the forefront of noteworthy Florida initiatives such as 
Tech Prep, High Schools That Work, and Blueprint for Career Preparation when they were funded. The NMHS 
schools were viewed as committed to developing their capacity to manage and sustain change (Drawer, 1 998). 

Funding for the schools ranged from $100,000 to $200,000 the first year, with a second disbursement of $50,000. The 
NMHS sites were charged with becoming prototype systems with a relevant, work-based curriculum that integrated 
career guidance and sponsored strong business partnerships. The millennium theme was created from one of Florida’s 
goals for reform and accountability of a high-quality school system (http://www.fim.edu/doe). 

3. The FCAT is based on the Sunshine State Standards. 

4. One respondent referred to a report that was developed by the millennium schools for the state during the first year 
of funding. The schools involved in this study shared their new curriculum guides (described holistically herein) but 
no such reports were provided. 
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Resumen 

La globalizacion impacta significativamente sobre las formas de gobierno las cuales, a su vez, 
crean una nueva institucionalidad que sirve de base para la transferencia de politicas publicas, En 
este articulo se utiliza el Marco de la Transferencia de Politicas para analizar el caso de la 
educacion superior en Mexico, y se sostiene que historicamente el sistema universitario 
mexicano ha usado de manera recurrente la experiencia externa para la elaboracion de sus 
propias estrategias en materia educativa, Asimismo, se destaca que: (1) el analisis general sobre 
la adopcion o copia de politicas educativas en Mexico se ha centrado basicamente en el papel que 
desempenan algunos organismos intemacionales lo que genera una vision limitada de la 
transferencia de politicas; (2) cs dificil identificar que objetos se estan realmente transfiriendo de 
estos organismos a la agenda educativa nacional y, por lo tanto, (3) validar la existencia de una 
transferencia de politicas educativas es un proceso complicado y abierto a cuestionamientos de 
tipo metodologico. Por lo tanto, se sugiere la realizacion de estudios futuros con el proposito de 
comprender mejor la adopcion o copia de politicas educativas en Mexico, 

Abstract 

Globalization and governance pave the way for the ocurrence of policy transfer processes. This 
article shows that higher education policy in Mexico has been historically influenced by foreign 
educational m.odels. By using the Policy Transfer Framework, this article also makes three 
claims: (I) the academic debate has been generally focused on the role of specific inlemational 
organizations, so a limited perspective about the policy transfer approach emerges; (2) the 
process of identifying what objects are being transferred to Mexican educational agenda is 
blurred, and thus, (3) as a consequence of methodological weaknesses, it is difficult to validate 
whether or not policy transfer processes are taking place in this Latin American country. 
Therefore, further research needs to be done in order to broaden the understanding of the policy 
transfer processes in Mexico, 
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La copia o adopcion de poHticas de educacion superior de un contexto a otro ha sido un tema de recurrente polemica y debati 
que para algunos academicos la reforma universitaria mexicana de la decada pasada respondia a “los imperatives extemos” ( 
1998:19), el gobiemo se abocaba a modemizar la educacion de acuerdo con la nueva realidad global. La modemizacion educ 
decia, era “inevitable y necesaria” (PME, 1989:iii). Esta divergencia muestra una amplia diversidad de puntos de vista que hi 
por una parte, el surgimiento de la globalizacion, la reduccion de las funciones del Estado y la poderosa influencia de organic 
intemacionales con la reformulacion de la agenda educativa en paises como Mexico. Sin embargo, a pesar del analisis critico 
realizado diversos academicos (Marin, 1998; Coraggio, 1998; Latapi Sarre, 1998; 2000a, 2000b; Alcantara, 2000; Rodriguez 
Alcantara, 2001 ; Gonzalez Casanova, 2001), aun existe una vision limitada de la transferencia de politicas educativas en Me: 
proposito de este articulo es ampliar la comprension de estos procesos de adopcion de acciones encaminadas a reformar la ed 
superior. En la primera parte se estudia la globalizacion como un fenomeno que presenta ciertas caracteristicas que promuevj 
interaccibn entre actores e instituciones y que, a su vez, facilita el ambiente para que se den transferencias de politica de un c 
otro. Este fenomeno mondial tambien ha transformado significativamente el papel del Estado-nacion como unidad predomin: 
forma de gobiemo. Por esta razon, la segunda parte analiza la recomposicion del Estado segun varios autores y se subraya el 
surgimiento de una nueva institucionalidad global con actores que interactuan en un ambiente de creciente complejidad. En 1. 
parte, se explica el enfoque de la transferencia de politicas, y su relacion con la globalizacion y con los distintos elementos qi 
componen. A pesar de la utilidad que este esquema analitico provee, su perspectiva teorica ignora en cierto grado la realidad 
paises pobres, por lo que habria que extender esta vision con mayores estudios. Para ello, se propone una clasificacion de los 
organismos intemacionales con base en sus caracteristicas particulares, medios de influencia o presion y los destinatarios de 
acciones. 



En la cuarta y ultima parte se presenta el caso de las politicas educativas a nivel superior en Mexico, y se hace un repaso hist 
correspondiente politica educativa, lo que muestra que el sistema universitario mexicano ha estado influido y, por lo tanto, hi 
adoptado elementos de sistemas academicos extranjeros. Sigue a este analisis, el estudio de la supuesta transferencia de polit 
educativa en este pais, haciendose tres observaciones: 1. El analisis general se ha centrado primordialmente en el papel y en 1 
de presion que utilizan los organismos intemacionales para influir en la agenda educativa nacional; 2. Es dificil identificar lo 
tangibles e intangibles que realmente se estan transfiriendo a la agenda educativa mexicana; y 3. Validar la existencia de un \ 
transferencia de politica educativa en Mexico es un ejercicio complicado y esta abierto a diversos cuestionamientos. A1 comf 
algunos documentos de politica educativa a nivel superior de Mexico con los informes de dos organismos intemacionales (el 
Intemacional para el Desarrollo de la Educacion, CIDE y la Organizacion para la Cooperacion para el Desarrollo Economico 
se sostiene que, varios anos antes, las autoridades mexicanas ya habian planteado acciones muy similares a las que el CIDE ) 
sugirieron en su momento para resolver los problemas de la educacion universitaria en Mexico. 

A1 no haber, por un lado, una clara evidencia de que Mexico copia la politica educativa de algunos organismos intemacionali 
determinado periodo y, por otro, saber que existen coincidencias entre programas nacionales e informes intemacionales, surg 
la pregunta, ^como se producen estas convergencias? Si se asegura que son las macroestructuras (globalizacibn, sistemas eco 
etc.) las que imponen estos cambios y los hacedores de politica aprenden de ellos y los copian, entonces se asume que existe 
ejercicio consciente y voluntario para copiar o emular una politica. Si es asi, ^que papel desempenan, entonces, los organism- 
intemacionales? Estas instituciones, junto con sus variados mecanismos de presion e influencia, son una parte clave (pero no 
para copiar o emular determinadas politicas dentro de un ambiente de creciente complejidad; por ello, es necesario ampliar h 
comprension de la transferencia de politicas educativas con estudios futuros. 

I. Lo nuevo del fenomeno globalizador. Un acercamiento al escenario de la complejidad 



Varios autores coinciden en senalar que la globalizacion no es un fenomeno nuevo, aunque el proceso de interaccion mundia 
tiene algunas variantes caracteristicas de la epoca. (Note 1) Por ejemplo, el Programa de Desarrollo de la Naciones Unidas (F 
1999) sostiene que la globalizacidn de este tiempo recuerda a la de los siglos XVI y XIX, solo que ahora existen “nuevos me 
ligados globalmente; “nuevas herramientas” como internet y los telefonos celulares. El PDNU tambien observa que existen “ 
actores” como la Organizacion Mondial de Comercio (OMC) que tienen autoridad sobre los gobiemos nacionales; las transni 
con un poder economico mayor al de muchos estados, las redes globales de organizaciones no gubemamentales y otros grup< 
trascienden las fronteras. Ademas, ahora hay “nuevas reglas” como los acuerdos multilaterales de comercio, servicios y prop 
intelectual que estan avalados por fuertes mecanismos de coaccion. Todo esto, segun el PDNU, forma un complejo de mayor 
restricciones para los gobiemos que cuentan con un reducido margen de maniobra para el desarrollo de politicas nacionales. 



Por su parte. The Economist argumenta que la integracion economica que se vive actualmente tiene sus antecedentes en siglo 
por lo que “la muy difundida idea de la globalizacion de estos dias es un regreso a tendencias pasadas” (1999:5). Segun The . 
cincuenta anos antes de la primera guerra mundial existieron flujos de mercancias, capital y personas que cruzaban la fronter 
distintos paises. Sin embargo, este proceso se interrumpio abruptamente para dar paso al proteccionismo y a las restricciones 
movimiento del capital. Indagando sobre cual sera el futuro de la globalizacion. The Economist se pregunta si este proceso p< 
revertido al igual que otras tendencias. La conclusidn a la que llega el semanario ingles es que sera muy dificil que esto ocun 
por ejemplo, los tratados de libre comercio actuales se elaboran sobre bases institucionales mas fumes que en el pasado. Mie 
otro tiempo, el libre comercio se formalizaba mediante tratados bilaterales; ahora se hace a traves instituciones multilaterales 
OMC {The Economist 1 999). 



Como puede observarse, tanto el PDNU como The Economist coinciden en senalar que la globalizacion no es un fenomeno n 
que parece caracteristico de esta epoca es que ahora existe una nueva institucionalidad que sustenta este proceso. Segun el PI 
intereses econdmicos ahora se mueven a traves de un amplio espectro de diversos actores que toman acuerdos y que pueden « 
influencia sobre los gobiemos nacionales. Este escenario no deja de ser complejo, por ende, es inevitable pensar en la nueva 
reorganizacion de los sistemas de gobiemo en estos tiempos. La globalizacidn, sostiene Cemy, “desafia profundamente las e: 
estructuras socioculturales y poHticas” (1999: 190). Por esta razdn, interesa analizar cdmo ha cambiado la forma de gobemar. 



especiTicamente, como se ha transformado la funcion del Estado-nacion dentro de la globalizacion. 

II. Gobierno y globalizacion. Pasos hacia una nueva institucionalidad 

En el contexto globalizador actual, el Estado - como unidad predominante de forma de gobierno - y sus funciones han sido a 
bajo diferentes perspectivas. 



For una parte, estan aquellos academicos que ven al Estado-nacion como el “actor por excelencia” (Marin, 1998) mientras qi 
encuadran el papel del Estado dentro de la perspectiva de la dependencia. For ejemplo, para Torres y Fuiggros, “el estado qu< 
promueve la modemizacion de la economia y de la politica sigue siendo un estado capitalista de desarrollo dependiente” (19! 
esta perspectiva, Ines Castro agrega que, bajo el “neoliberalismo”, el estado “ya no es visualizado como el instrumento idone 
superar las diferencias sociales” (1997:195). Jose Luis Coraggio relaciona al Estado, con la funcion de poderosos organismoj 
financieros como el Banco Mondial (BM), asegura que este organismo se convirtio en el “Caballo de Troya” del ajuste estrut 
la redefinicion del papel del mercado y del Estado en el campo de las politicas sociales y agrega: “En las practicas de esos or 
es evidente un claro interes estrategico asociado al paradigma neoliberal que comparten: restituir al mercado el lugar que ten: 
estas decadas de estatismo.” (Coraggio, 1998:50-51) 



For su parte, Gonzalez Casanova afirma que el poder del sector privado actuo como fuerza para desmantelar o debilitar al se( 
publico de los paises comunistas, de los estados benefactores socialdemocratas y de los estados populistas” (2001 :22). Algun 
como Castro, Coraggio y Gonzalez Casanova coinciden en que existe una reduccion del papel de los estados-nacion dentro d 
escenario mondial; sin embargo, cada uno menciona causas diferentes que podrian explicar el “ahuecamiento” (Note 2) estat; 
Mientras Castro apunta que el “neoliberalismo” es el responsable de la reconfiguracion del estado, Coraggio subraya el “inte 
estrategico” de los organismos financieros en los actuales cambio estatales y, Gonzalez Casanova defiende la tesis de que exi 
“empresariato corporativo de las potencias hegemonicas” para apuntalar la politica “macrocultural y macroeconomica de 
reestructuracion nacional-global” (2001:24). 



Asi, la modemizacion, la dependencia, el “neoliberalismo”, el ilimitado poder de los organismos intemacionales y demas fue 
parecen ser elementos que algunos academicos usan para explicar la reconfiguracion del Estado dentro de la globalizacion. L 
de vista de estos academicos difieren, en cierto grado de los de otros estudiosos, por ejemplo, politologos como RJiodes (199 
aseveran que desde hace quince anos han surgido terminos para reformar el sector publico, tales como governance (Note 3) c 
sustituye al termino government y explica las caracteristicas del governance, diciendo que dentro de este “si sterna gubemami 
existe una interdependencia de organizaciones y que el governance es mas amplio que el gobierno e incluye actores no estata 
Asimismo, hace hincapie en que las fronteras entre el sector publico, el sector privado y el voluntario son menos visibles por* 
cambiado las fronteras del Estado mismo (Rhodes, 1997:53). Otra caracteristica que menciona Rhodes es la continua interac< 
los miembros de las redes de politica publica, (Note 4) causada por la necesidad de intercambiar recursos y negociar interese 
y que debe basarse en la confianza y en la regulacion que comparten los participantes de las redes. La ultima caracteristica es 
de autonomia con respecto al Estado o sea que las redes de politica no tienen responsabilidad con respecto al Estado, pues es 
autoorganizativas; aunque el Estado puede controlarlas de manera “imperfecta” e “indirecta” (Rhodes, 1997:53). Es importai 
que, para Rhodes, gobemar se asemeja a un juego que tiende a ser “entre iguales”, es decir, con actores con intereses similan 
sus caracteristicas varien. Asimismo, se advierte que en el governance puede haber interdependencia e intercambio, pero tarr 
reglas. Fueden existir intereses comunes al igual que confianza y, aun mas importante, puede haber autonomia de estas redes 
respecto al Estado, aunque tambien hay control. 

Llama la atencion que desde la perspectiva de Rhodes no se otorgue un peso preponderate a los organismos intemacionales 
parte de las redes de politica. Ademas, llama la atencion que en su libro Understanding Governance, Rhodes (1997) no utilic 
termino “neoliberal” para referirse a la privatizacion o a la competencia. Estas politicas, subraya, son “distintivas” del Fartidi 
Conservador Britanico. 

En otro tenor y, tratando de visualizar el fenomeno de la globalizacion en una dimension mas amplia, con respecto al goverm 
Cemy piensa que la globalizacion impacta sobre el governance “modificando las estructuras en donde se desarrollan sus pro( 
mecanismos” (1999: 188). 

Este autor sostiene tambien que se modifican condiciones o parametros que afectan la posible combinacion de la jerarquia, e) 
y las redes y esto puede ser, segun Cemy, dentro de las estructuras socioculturales, de la produccion economica y del consun 
como en los procesos politicos e institucionaies (1999). Cemy provee una vision holistica de la forma en que la globalizacior 
transformar los sistemas de poder, el cual “inevitablamente” se hara mas difuso y permeara una estructura “compleja y prism 
fuerzas socioeconomicas y niveles de governance (1999:190). Dentro de esta vision integral de fuerzas exogenas y poder, an; 
“El mundo esta ahora mas inmiscuido en un proceso de globalizacion compleja y lo mas urgente, dentro de una agenda de 
investigacion tanto en relaciones intemacionales como en ciencia politica, no solo en el corto sino tambien en el largo plazo < 
identificar las numerosas dimensiones de este proceso complejo y evaluar las estructuras de las intersecciones e interaccionej 
ellas”(Cemy 1999:209). 

Segun Cemy, el sistema estatal han sido visto como formas de gobierno en el nivel macrosocial; no obstante, su coexistencia 
formas de gobierno a nivel microsocial, mesosocial, transnacional e intemacional, y es la globalizacion la que promueve el si 
y cristalizacion de nuevas formas de gobierno (governance). Esta vision parece diferir de aquella que postula al Estado-nacio 
“actor por excelencia” (Marin, 1998). 






Otro autor importante que habla de la recomposicion de los estados-nacion y sus funciones es Anthony Giddens, quien sostie 
globalizacion, en lo que el mismo ha llamado la tercera via, “conlleva una necesidad de mas gobiemo y no menos” y agrega * 
“Aunque el gobiemo ya no puede ser simplemente nacional. Tiene que haber un gobiemo local y regional fuerte, pero hay qi 
tambien formas de gobiemo transnacional (en El Pais, 25/07/99) (Note 5) . 



Como puede observarse, los centros de poder e influencia politica se han cambiado a consecuencia de la globalizacion y segi 
bajo este escenario la toma de decisiones sera mas compleja de lo que sucedia cuando el Estado era el “actor por excelencia” 
las palabras de Marin). En congruencia con este punto, Mundy y Murphy (2001) observan que aparte del extremo poder de k 
organismos financieros, estan apareciendo nuevos actores intemacionales no gubemamentales, tales como las Redes Defense 
Transnacionales {Transnational Advocacy Networks. TANs) o los Movimientos Sociales Tran snaci on ales {Transnational Soc 
Movements, TSMs) que tienen repercusiones sobre el desarrollo del sistema politico mondial y que representan una “nueva cl 
activismo no gubemamental” (Mundy y Murphy 2001:88). Segun estos autores, actualmente, estas redes estan al margen del 
estatal, toda vez que ya no dependen de los recursos proveidos por los estados-nacion. Es importante tomar en cuenta que la 
independencia que estas redes pudiesen tener se deriva de una transformacion en las funciones del Estado. Esta reduccion po 
cierto punto, crear espacios de mayor libertad para la actuacion de algunas fuerzas contestatarias. 

Hasta aqui, se ha analizado que el papel que desempena el Estado en el diseno e implementacion de politicas sociales se ha n 
considerablemente y que una nueva institucionalizacion global esta generando un intricado ambiente de intereses y actores qi 
relacionesde intercambio, conflicto, “resistencia” y “acomodo” (Note 6) cada vez mas complejas. Esta complejidad parece 
interpretarse desde dos perspectivas. Mientras algunos autores como Gonzalez Casanova hablan de “debilitamiento y 
desmantelamiento” del sector publico; Rhodes dice que hubo una “reforma” de este tipo de servicios; cuando Cemy subraya 
Estado interactua en diversos niveles de gobiemo, Marin considera al Estado-nacion como “actor por excelencia”. En tanto C 
Casanova piensa que se inicio una “reforma neoliberal y privatizadora de la educacion superior en Mexico”, Rhodes dice que 
privatizacion y la competencia son acciones conservadoras. (Note 7) 



Bajo estas dos perspectivas, no seria ilogico esperar que la interpretacion de algun proceso de adopcion de politicas publicas 
institucion o de un pais extranjero a otro sea diversa. Ambas formas de estudiar la realidad podrian ser capturadas dentro de i 
esquema analitico que ofrezca explicaciones mas solidas acerca de la adopcion, copia o influencia externa que pudieran ejerc 
instituciones o paises sobre la politica publica de otras naciones. Por ello, el enfoque de la transferencia de politicas podria sc 
altemativa en este sentido; por lo que, a continuacion, se explicaran algunos de sus elementos sustantivos. 

111. £1 enfoque de la transferencia de politicas 

A. Relaci6n entre globalizacidn y transferencia de politicas 

En parrafos previos se anot6 que la globalizacion ha modificado el papel del gobiemo y que una institucionalidad a escala mi 
emergiendo; por lo que tambien puede pensarse que la transferencia de politicas (TP) se convierta, como senala Common, en 
“manifestacion cada vez mas visible” (2001:5). De hecho, Evans (1999) opinaque hay tres razones por las cuales ha habido i 
creciente interes en el estudio de la transferencia de politicas publicas. Primero, a consecuencia de los procesos de globalizac 
extemos al Estado-nacion (integracion politica y comunicacion global) asi como a los intemos (el “ahuecamiento” que sufre 
a travds de, por ejemplo, privatizaciones), se han creado nuevas “estructuras de oportunidad” para la transferencia de politica 
segunda razon es porque es mucho mas probable que la transferencia de politicas ocurra en una era en donde exista el govern 
agrega que ante la incertidumbre que pudieran tener los hacedores de politica y ante el apremio de tiempo, la TP puede dar ui 
solucibn rapida a los problemas de politica publica. Evans acota que la tercera razon para promover los procesos de TP es qu 
habido una relacion muy cercana entre las administraciones de George W. Bush y del primer ministro britanico Tony Blair, > 
ejemplos de ello en materia de educacion, de justicia y de reformas financieras (1999). 

A pesar de ser una perspectiva britanica, esta vision de Evans conduce a pensar que la TP podria ser utilizada bajo el esquem 
cooperacion intemacional que ha sido un discurso recurrente dentro de la globalizacibn y a considerar la utilidad de la TP pu 
premura e incertidumbre, este esquema podria usarse como una altemativa para disenar o cambiar una politica publica. Con < 
ahorrarian recursos materiales y monetarios, y ademas, como apuntan Heidenheimer et al (1976), se comprenderian mejor lo 
problemas al estudiarlos de manera compartida y comparada. Claro, estas ventajas no invalidan el hecho de realizar adaptaci* 
acriticas de un modelo exterior a una realidad especifica. 



En resumen, la globalizacion y la nueva institucionalidad mondial facilitan el terreno para que surjan procesos de transferenc 
politica, por eso es importante saber por que y como se dan estas iniciativas, quienes las promueven, que es lo que se transfie 
traves de que medios y como puede verificarse que hay una copia o adopcion de politica publica de un contexto especifico a 
esto ponderando tanto la utilidad como las desventajas de estos procesos. 

B. Definicibn de la transferencia de politica publica 



Dolowitz y Marsh aluden que: “La transferencia de politicas, la emulacion y leccibn aprendida se refieren a un proceso en el 
conocimiento acerca de politicas, sistemas administrativos e instituciones, etc., en determinado momento y/o lugar, es utiliza 
desarrollo de politicas, sistemas administrativos e instituciones en otro momento y/o lugar ( 1996:344). 

Evans complementa la definicibn de Dolowitz y Marsh al aseverar que “La transferencia de politica es un concepto generico 
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significa un proceso en el cual el conocimiento acerca de las instituciones, poHticas o sistemas de suministro en un sector o n 
gobiemo (governance) es usado en el desarrollo de instituciones, poHticas o sistemas de entrega en otro sector o nivel de 
gobiemo.** (1999:30). 

Es importante notar el career “generico’* del concepto de transferencia de poHtica que menciona Evans, ya que este puede a 
“afirmaciones muy distintas acerca de la naturaleza de la elaboracion de poHticas” (Evans y Davies, 1999:204). “Leccion 
aprendida” (Rose, 1991), “difusion de poHtica” (Majone, 1991) o “importacion de poHtica (Note 8) ” (Brown, 1999) podn'an 
algunas de estas afirmaciones que se refieren a un proceso de aprendizaje en el cual los hacedores de poHtica publica observe 
el mundo las respuestas de sus contrapartes a problemas similares” (Mokhtar, 2001:22) y, entonces, disenan sus propias polit 

C. Segmentaci6n y niveles de transferencia de poHticas publicas 

Se ha observado que sobre las transferencias de poHtica publica diversos academicos como Bennet (1991) Wolman, (1992), 1 
Evans (1997) estudian la emulacion de poHticas, programas gubemamentales e instruments y esquemas de administracion, 
exclusivamente en los paises desarrollados. Sin embargo, para entender de manera mas integral la TP, es necesario hacer una 
segmentacion entre paises en desarrollo y los de mayor avance economico, ya que las estructuras estatales, los actores y la 
institucionalidad difieren significativamente entre paises pobres y ricos. Esto no significa que la TP se de solamente de una n 
otra, ya que como apuntan Evans y Davies, existen cinco niveles de “territorialidad poHtica” (transnacional, intemacional, na 
regional y local) y dentro de este marco espacial, por tanto, existen 25 trayectorias teoricas que pueden seguir las 
transferencias” (Evans y Davies, 1999: 212) (vease el cuadro 1). 



Cuadro 1. Trayectorias de la transferencia de politicas 
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Intemacional 



Intemacional 



Intemacional 
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Local 



Fuente; Evans y Davies (1999:213). 
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A pesar de que Evans y Davies (1999) esquematizan las trayectorias que podn'an seguir las transferencias, tambien senalan c» 
sensatez que “no es probable” una transferencia, por ejemplo, entre estados federales en Alemania o entre regiones britanicas 
gobiemo regional no tiene categon'a oficial. Es decir, en muchos casos no existen condiciones para que alguna de estas traye< 
puede darse. Por ejemplo, en un pais cuyos estados esten divididos en municipios, una experiencia desarrollada en alguna de 
local idades no podra ser emulada por otra nacion cuya division poHtica sea por regiones autonomas o por condados. Por lo t£ 
trayectoria “locaHintemacional” esta lejos de ocurrir y, sobre todo, lograr el impacto esperado. 



A pesar de la improbabilidad de algunas trayectorias mencionadas por Evans y Davies, no puede negarse que su idea sobre h 
diferentes trayectorias ampHa la comprension de los procesos de transferencia de politicas. 

D. Tipos de transferencia de poHticas 

Describir los tipos de transferencias de poHtica sirve, en parte, para conocer los actores y los objetos que se supone se transfi< 
Dolowitz y Marsh (1996) identifican tres tipos generales de TP. El primero se refiere a un proceso voluntario en el cual exist 
decision de los gobiemos para emular estrategias gubemamentales de otros contextos. Este tipo de transferencia se origina, s> 
(1991), por dos causas principales: i) la insatisfaccion en el desempeno de ciertas poHticas y/o sus resultados, y ii) la incertid 
acerca del futuro. Dentro de esta ultima causa, pueden encontrarse tambien dos niveles de incertidumbre. Aquella relacionad: 
nivel macro — como podria ser una crisis financiera — y la micro, que toma en cuenta asuntos de ambito local o regional — cor 
ser las elecciones competidas — . Hablar de insatisfaccidn o incertidumbre plantea el cuestionamiento, ^como puede saberse c 
una transferencia de poHticas de tipo voluntario como consecuencia de alguna de estas dos razones o, quiza, por la combinac 
ambas? Bennet (1991) senala que hay poca o nula atencion a la validez metodoldgica para conocer cuaies son las motivacion 
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de los hacedores de politica para adoptar o emular alguna politica de algun contexto extemo. Quiza un mecanismo de indaga 
las entrevistas con funcionarios o la revisidn documental, para saber las causas que motivaron la TP. Este hecho muestra la d 
de validar los procesos de transferencia de politicas. 

El segundo tipo de TP se caracteriza por implicar una forma coercitiva que se basa en la unilateralidad del emisor para dictar 
recomendaciones al receptor. Segun Dolowitz y Marsh (1996), la imposicion directa de TP de un pais a otro es “rara”; sin err 
estos autores identifican a los organismos supranacionales como el Banco Mondial (BM) y el Fondo Monetario Intemaciona! 
como instituciones clave en la transferencia coercitiva de politicas. Estos autores subrayan que el medio de coercion son los ] 
que se otorgan a los paises pobres, los cuales tienen que seguir ciertas politicas economicas si estas naciones quieren hacer uf 
recursos prestados. Las empresas transnacionales (TNCs, por sus siglas en ingles) tambien pueden operar dentro de la transfe 
coercitiva, ya que pueden forzar a los gobiemos hacia la transferencia de politicas porque poseen un “as bajo la manga”. La 2 
de llevarse sus negocios a otra parte presiona a los gobiemos a adoptar politicas para atraer a este tipo de industrias, segun D- 
Marsh (1996:348). 

El tercer tipo es la transferencia coercitiva indirecta en la que, tedricamente, no intervienen determinados agentes para motiv; 
cambio de politica publica. Dolowitz y Marsh delimitan cuatro factores estructurales dentro de este tipo de TP: i) Las externa 
ii) los cambios tecnoldgicos; Hi) la interdependencia economica, y iv) el desempeno actual de alguna nacion. Estos factores s 
resumir en lo que Mokhtar (2001) llama “el poder estructural de la economia global”, que presiona a un pais a adoptar politic 
simi lares a otros regimenes con los cuales puede compartir un mercado comun. La competencia, dice esta autora, “provoca u 
tendencia para la transferencia de politica” (2001 :33). Un ejemplo de este ambiente podria ser la formacion de mercados coir 
como el Tratado de Libre Comercio (TLC) para Norteamerica, en donde Mexico es el pais en mayor desventaja economica y 
por lo tan to, su gobiemo podria adoptar algun as politicas copiando lo que hacen sus contrapartes comerciales (Estados Unidc 
Canada). “El miedo de quedarse atras en cuestiones publicas puede motivar a la accion”, dice Bennett (1 991 :43). 

E. Agentes de la transferencia de politicas y su clasificaci6n 

Los actores son una parte crucial para la emulacion de acciones gubemamentales, ya que, como argumentan Davies y Evans 
relaciones entre estos pueden facilitar o precipitar los procesos de transferencia. Por su parte, Dolowits y Marsh identifican, c 
literatura relativa a la TP, seis actores clave: i) autoridades elegidas, H) partidos politicos, Hi) burocratas y servidores publico 
grupos de presion, v) emprendedores de politica y expertos, y vi) instituciones supranacionales. En el caso de las institucione 
supranacionales, Dolowitz y March incluyen tambien a las organizaciones intergubemamentales y a las agencias intemacion; 
como la Comun idad Europea y la Organizacion para la Cooperacion y el Desarrollo Economico (OECD)--; sin embargo, est( 
no hacen una clasificacion de estos agentes de la TP de acuerdo con sus fines, origenes, intereses y medios de influencia. En 
2, se presenta una clasificacion de los organismos intemacionales que pueden funcionar como agentes de la transferencia de ] 



Cuadro 2. Clasificacion de los organismos intemacionales 



Organizaciones 


Caracteristicas 


Medios de influencia 


Raises desti 


Intergubemamentales 
(ONU, UNESCO, OECD) 


Trascienden las fronteras. 
Creadas por acuerdos 
multi late rales. 

Asociacion voluntaria. 


Recomendaciones. 
Intercambio de informadon 
Esquemas de cooperacion. 


En desarrollo y 
desarrollados 


Intemacionales no 
gubemamentales 
{CruzRoja, Amnisti'a 
Intern a clonal) 


No existen fines de lucro. 
Ayuda humanitaria. 
.Autonomi'a de los estados. 


Reportes. 

Actividades alrededordel 
mundo. 


En desarrollo y 
desarrollados 


Supranacionales (FMI, BM 
yComunidad Economica 
Euro pea) 


Existen y operan sobre los 
Estados. 

Se abocan a temas 
especificos. 


Prestamos. 

Prescripcion de politicas. 


En desarrollo, desar 
y en transicion. 


Transnacionales (IBM, GM, 
etc.) 


Alta movilidad regional 
Presencia mundial 


Inversion externa 
Creadon de fuentes de 
empteo 


En desarrollo 
particulaimente. 



Los seis agentes que identifican Dolowitz y March tambien pueden clasificarse en tres grupos generales, de acuerdo con su fi 
primordial. En el primer grupo, estarian los individuos o instituciones que desempenan un papel oficial. Aqui se incluirian a ! 
servidores y funcionarios publicos, asi como a los representantes electos. En el segundo grupo, cabrian los actores que ofrece 
en politicas publicas y que poseen conocimiento especializado sobre temas espedficos como educacion, salud y politica mon 
etcetera. En esta categoria tambien se podria contar a los llamados think tanks. Por ultimo, en el tercer grupo, estarian los gru 
presion, partidos politicos, sindicatos, redes defensoras transnacionales {Transnational Advocacy Networks, TANs) y los mov 
sociales transnacionales {Transnational Social Movements, TSMs) que han surgido recientemente, segun observan Mundy y ' 
( 2001 ). 



F. Objetos de transferencia 
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^Cuales son en realidad los elementos que se transfieren de un lugar a otro? ^Acaso solo las poHticas publicas? Indudableme: 
Existen varios elementos que se movilizan de un contexto a otro y las poh'ticas solo son parte de un conglomerado. Dolowitz 
delimita cinco objetos en la TP: poHticas publicas, instituciones, ideologias, actitudes y lecciones erroneas. Como puede obsc 
existen elementos en la TP que son intangibles y dificilmente comprobables, tales como ideas y actitudes. Sin embargo, algu; 
autores como Roberts y Waltman ban mostrado que al adoptar una retorica ideologica, el ex presidente de los Estado Unidos 
Reagan y la ex primer ministro, Margaret Tatcher justificaron y diseminaron sus pro gramas politicos basados en la “nueva 
derecha” (citados en Dolowitz y March, 1996). 

Antes de analizar otro elemento de la TP, es necesario decir que la educacion puede ser tanto un objeto de transferencia comi 
para di seminar, entre otros, ideas o ideologias. Es decir, se podn'an transferir desde modelos o poHticas educativas de un luga 
asi como tambien se podria usar el conocimiento y a sus portadores como medio para diseminar ideas, creencias y valores. Pi 
ejemplo, un alto funcionario del Ministerio de Educacion de Canada pensaba que tener una economia avanzada dependia de ! 
“transfertilizacidn de ideas”, por ello era fundamental para ese pais que los canadienses se prepararan en el exterior y que pei 
otros paises llevaran su talento a ese pais (note de Claudia Herrera en La Jornada, 28.10.99). Este punto no debe confundirse 
hecho de que la educacion reproduce cierta ideologia pues entonces, como dice Paulo Freire, no habria porque apelar en la pi 
educativa a la capacidad critica del individuo (1996). Lo que es interesante es observar que en los procesos de TP tiene un ca 
ambivalente y esto es un campo fertil para realizar an^isis en el futuro. 

G. Grades de transferencia 

Dolowitz y Marsh (1996) enumeran cuatro diferentes grados de transferencia de politicas que varian de acuerdo con los caml 
adecuaciones que se hagan a ’’las lecciones aprendidas”. El primer grado es copiar una politica publica y este es el grado mas 
que existe, ya que no se realiza ningiin cambio sustantivo a la politica importada. El segundo grado es la emulacion en la que 
no copia una politica especifica, sino que la usa como base para el disefio de su propia politica. Para Rose (1991), la emulaci» 
aceptacion en primera instancia de algiin esfuerzo gubemamental del exterior que requiera cierto grado de adaptacion al nue\ 
contexto. El tercer grado es la hibridacion, que implica una combinacion de elementos de las poHticas de varios casos para ui 
adaptacion al contexto receptor. Y el ultimo grado es la inspiracion, que surge de la observacion de problemas particulares pj 
el desarrollo de algun programa o politica publica. Este grado de transferencia es, segiin Rose (1991), solo una forma de espe 
posible resultado en un contexto diferente aunque no representa, logicamente, una transferencia. 

Tomando en cuenta estos diferentes grados de transferencia, seria interesante saber hasta que punto alguno de el los ha results 
mas efectivo que otro. Desafortunadamente, la evidencia empirica hasta el momento no es suflciente para contestar esta preg 

H. Elementos que facilitan y limitan las transferencias 

Dolowitz (1997) especifica seis categorias generales que pueden facilitar o restringir los procesos de TP. La primera es la coi 
de la politica publica: cuanto mas compleja sea la politica o el programa, tanto mas dificil sera transferirla. Por lo tanto, exist 
relacion entre la complejidad del programa o politica y su nivel de transferencia. El segundo aspecto tiene que ver con el aspi 
adaptacidn de la politica o programa a transferir, es decir, la interaccion entre una politica adoptada y una politica existente o 
con anterioridad. La tercera categoria es la de limitantes de las estructuras institucionales. Dolowitz establece que si las estru- 
institucionales difieren radicalmente, las posibilidades de que exista una transferencia son muy limitadas. En este rubro, taml 
menciona un punto de suma importancia, que es el sistema legal de ambos contextos: el emisor y el receptor. Es de esperarse 
diferente tipo de normativa, los procesos de transferencia tardaran en ocurrir o definitivamente no se llevaran a cabo. 

La cuarta categoria incluye varias subcategorias, como ideologia politica, semejanza cultural, capacidad tecnologica y recurs 
economicos que podrian existir entre el emisor y el receptor. La quinta categoria es el lenguaje que, segiin Dolowitz, ayuda a 
acceso a la documentacion requerida en el diseno de politicas, asi como el analisis personal de las mismas. La sexta category 
ver con las relaciones anteriores que existen entre los sistemas politicos. Dolowitz ejemplifica este punto resaltando las relaci 
antagonicas que pudieran existir entre algunas unidades dentro de los gobiemos, y dice que las relaciones “armoniosas”, deni 
fuera de las fronteras ayudaran a establecer las lineas de comunicacion necesarias para una efectiva transferencia de politicas 

Por ultimo, es necesario destacar que autores como Dolowitz (1997) y Evans (1999) mencionan la afinidad entre los gobieiru 
estadounidenses y britanicos. Estas semejanzas son un ejemplo de que ambas naciones comparten una ideologia politica simi 
cultura comiin: la anglosajona. Ademas, tienen una capacidad tecnologica comparable y cuantiosos recursos economicos; en 
paises se habla ingles y sus relaciones diplomaticas han sido caracterizadas historicamente por un ambiente de armonia y coc 

Validaci6n de la transferencia de poHticas 

Una de las debilidades que el esquema de la transferencia de politicas posee es su dificultad para ser validado. Algunos autor 
(Dolowitz y Marsh, 1996) ex ponen varias estrategias en este sentido, tales como la revision de reportes, programas gubemar 
visitas, entrevistas, etc., con el proposito de indagar si realmente se transfiere un programa o politica de un lugar a otro. Por s 
Evans y Davies (1999) proponen una metodologia basada en cinco bloques de preguntas para validar la existencia de los casi 
transferencia de politicas. A saber: 

• ^Cual es el sujeto de analisis? Es importante, dicen Evans y Davies, ser claros acerca del fenomeno que se estudia, y 
trabajo empirico “primario” existe una gama de posibilidades (1999:236). 

• ^Quien o que es identificado como agente o agentes de la transferencia? ^quien la desea? ^que se quiere obtener con 




^como la estan llevando a cabo? ^en beneficio de quien? Y ^por que? Con referenda a estas preguntas los autores pi- 
la transferencia debe ser un proceso conciente, ya sea que se lleve a cabo voluntariamente o este sujeto a coercion (E 
Davies, 1999:237). 

• pruebas de no transferencia? Para contestar esta pregunta, Evans y Davies sugieren realizar una comparacion d 
la poHtica en cuestion con entomos nacionales y originales, si se desea determinar el grado real de transferencia en u 
particular {ibidem). 

• son las pruebas presentadas para apoyar la afirmacion? ^Que tan convincentes son? Estas preguntas se deriv 
Evans y Davies coinciden con Dolowits y Marsh (1996) en el sentido de que las entrevistas no demuestran de maner 
concluyente que la informacion proveniente de un sistema extranjero influyo en cuestiones nacionales y agregan: “[5 
argumenta que se ha copiado un programa, se espera que se presenten pruebas “fisicas’* mas concretas que apoyen ei 
declaracion. S61o se puede decir que un programa ha sido copiado por otro si se han comparado ambos. La cuestion • 
programa se ha realizado podria ser entonces sometida al analisis de la implementacion para determinar que tan lejoi 
llegado la transferencia (Evans y Davies 1999:237-8)” 

• lQ\^^ conclusiones se pueden sacar de lo expresado antes acerca de la naturaleza y el grado de la transferencia que S( 
producido? ^Son ideas o programas implementados los que han o no han sido transferidos? Si se encuentra que se ha 
producido la transferencia, entonces se podria determinar el grado de esta copia, emulacion, hibridacion, sintesis o ir 
(Evans y Davies, 1999: 23). 

Las preguntas planteadas por Evans y Davies demuestran lo complejo que es validar un proceso de transferencia de politicas. 
puede observarse en la pregunta cuatro, estos autores proponen una escala que muestre si la transferencia ha ocurrido o no, y 
primero habra que saber si la transferencia se ha hecho a nivel declarativo. Esto implica comparar los documentos “extemos’ 
pudieran prescribir una transferencia de politicas con documentos y programas oficiales. En segundo lugar, habra que investi 
politica o programa supuestamente transferido fueron puestos en practica para saber “que tan lejos ha llegado la transferencia 
y Davies, 1999:238). Otro punto que seria interesante incluir, y que no mencionan Evans y Davies, es la evaluacion de los ef 
haber transferido una politica o un programa de un lugar a otro. Todos estos aspectos, sin duda, representan nuevos retos pan 
investigadores y hacedores de politicas publicas. 

Con el proposito de contribuir al estudio de la transferencia de politicas, en la siguiente parte se analiza el caso de las politica 
educativas en Mexico, especificamente a nivel superior. En fechas recientes, el sistema universitario ha sido objetode intens< 
Mientras que, para algunos, la reforma universitaria de la decada noventa respondia a “los imperativos extemos” (Marin, 19S 
gobiemo mexicano se abocaba a modemizar la educacion de acuerdo con la nueva realidad global. La modemizacion educati 
“inevitable y necesaria”, apuntaba el Programa de Modemizacion Educativa 1989-1994 (PME, 1989:iii). Este debate incluye 
vista que han ligado el surgimiento de la globalizacion con la reformulacion de una nueva agenda educativa en Mexico. Asin 
ha observado la poderosa influencia de organismos intemacionales, especificamente del Banco Mondial, en la formulacion d 
y programas educativos. Esto ha sido campo fertil para el examen critico de algunos academicos que observan las contradicc 
sus recomendaciones y la carencia de “precision teorica, metodologica o cientifica” de las ideas del BM (Coraggio, 1998:50) 
embargo, el analisis desarrollado hasta el momento ha carecido de un esquema analitico que amplie el entendimiento y la coi 
de la transferencia de politica educativa. Por lo tanto, en seguida se hara un repaso historico y se revisaran los argumentos ac 
sobre la supuesta transferencia de politica educativa superior en Mexico. 

IV. Analisis historico y debate sobre la transferencia de politica educativa en Mexico 

Aqui se ha sostenido que existe un renovado ambiente globalizador dentro del cual ha surgido una institucionalidad mas com 
cual ha modificado las formas de gobiemo. El Estado se ha transformado por una serie de factores extemos que, a la vez, inv 
nuevos actores en el disefio e implantacion de politicas publicas. Mexico dada su importante situacion geografica y economic 
del continente americano, no ha permanecido al margen de la globalizacion. Por ello, existe una gran probabilidad de sus hac 
politica utilicen el esquema de la transferencia de politicas para el diseno y reformulacion de acciones de caracter publico. Et 
de la educacion, copiar o emular una politica o un programa del exterior toma mayor relevancia pues este sector ha sido una 
politica social de los gobiemos posrevolucionarios (OECD, 1994). Por lo tanto, la idea de que pudiese existir injerencia extei 
definir la agenda educativa nacional podria abrir la puerta a severas criticas y a un intenso debate. 

Con el proposito de conocer si la politica educativa mexicana a nivel superior ha recibido una influencia externa mayor en es 
tiempos que en el pasado, cuando quiza no se conocia la palabra globalizacion, a continuacion se hace un breve repaso histor 
revisan los argumentos academicos sobre la supuesta transferencia de politica educativa en Mexico. 

A. ^Desde cudndo Mexico ha adoptado poHtica educativa extranjera? 

Cada etapa historica en el desarrollo de Mexico ha transformado y modificado las politicas educativas con el proposito de qu 
tengan mayor relevancia en terminos sociales, economicos o politicos. Tales transformaciones han provocado que se adapten 
educativos que directa o indirectamente han sido disenados con base en experiencias de otros sistemas academicos. Por ejem 
que recordar que a mediados del siglo XVI, cuando nacio la Real y Pontificia Universidad de Mexico esta adopto el sistema 
organ izacional y de gobiemo de la Universidad de Salamanca en Espana. M^s tarde y como apunta Carlos Tunnermmann (1< 
reforma de la universidad colonial se baso en el modelo napolednico. Este autor subraya algunas de las consecuencias de esU 
“eleccion” y escribe: “En vez de buscar la renovacion de los estudios a traves de la brecha abierta por los sabios americanos, 
constituia una respuesta original y hubiese conducido al arraigo de la investigacion cientifica entre nosotros, la Republica, tr{ 
pugnas entre liberales y conservadores por el dominio de la universidad que tuvieron lugar inmediatamente despues de la 
Independencia, no encontro mejor cosa que hacer con la universidad colonial que sustituirla por un esquema importado, el dc 
universidad francesa, ideado por Napoleon, tan a tono con el momento que se vivia de asombro ante todo lo que de Francia p 




(1996:20) 



Es interesante que Tunnermmann indique dos causas que impidieron el desarrollo de un proyecto universitario que tuviera, s< 
una “afirmacion nacional”. La primera es la pugna entre grupos de poder especiTicos (liberales y conservadores), y la segund 
“asombro” de los americanos por lo frances. 

Hacia finales del siglo XIX, la dictadura de Porfirio Diaz promovio en Mexico otra filosofia importada de Francia: el positivi 
desarrollado por August Comte. Esta filosofia, manifiestan Pallan et al (1995) se impuso en la ensenanza de las profesiones 
resultado de que la elite gobemante de ese tiempo estaba familiarizada con las ideas de Comte. En esos dias, una de las figun 
notables en materia educativa fue Justo Sierra, quien fungio como secretario de Instruccion Publica y Bellas Artes. Este horn 
idea de fundar la Universidad Nacional en 1910 la cual “tendria la potencia suficiente para coordinar las lineas directrices de! 
nacional” (citado en Paz, 1994: 159). Este caracter nacionalista de Justo Sierra parece confundirse cuando al crear la Universi 
Nacional, este prominente hombre tenia en mente un modelo de universidad que cristalizaba en la Universidad de Berkeley e 
Estados Unidos (Gil, 2000). Segun Manuel Gil esta institucion estadounidense contaba con profesores de tiempo completo qi 
en su mayoria, el grado de doctor, mientras que la naciente universidad mexicana carecia de ellos, por lo que Sierra confirio ; 
doctores a “numerosos academicos” que no eran “siquiera profesores de tiempo completo” (Note 9) para construir una univei 
“valiera la pena” (Gil, 2000:25). Como puede notarse, hubo puntos incompatibles entre lo que Justo Sierra imagine (una uni\ 
nacional), lo que conocia previamente (el sistema academico de Berkeley) y lo que tuvo a la mano para concretizar su idea (f 
mexicanos sin grades). Este es un claro ejemplo de la dificultad de poner en practica una noble idea y, de lo litil que resulta u 
experiencia extranjera en el diseno de politicas publicas. 

Dando un salto cronologico y ya situados en el periodo posrevolucionario, Pallan et al. (1995) notan que durante esa epoca s< 
promovieron actividades y carreras humanisticas para contrarrestar el positivismo diseminado en el sector educative del porf 
embargo, ese tiempo tampoco fue la excepcion para adoptar o emular modelos universitarios foraneos. La Reforma de Cordc 
1918, como dice Tunnermmann, se propago a lo largo y lo ancho de America Latina y fue el primer cuestionamiento serio di 
universidad latinoamericana tradicional. Para Tunnermmann fue un “movimiento original”, sin precedentes en el mundo, y q 
“la mano de la transformacion nacionalista y liberadora” que esta region necesitaba (1996:33) (Note 10) . 

Hasta aqui se ha observado que por diversas circunstancias, pugnas entre grupos gobemantes, moda, ideologias, experiencias 
tendencias, la educacion superior en Mexico recibio la influencia de modelos extranjeros. El escenario que surge a partir de 1 
economica de finales de los anos setenta y principios de los ochenta —crisis provocada tanto por incorrectos manejos de poli 
economica a nivel nacional como por problemas en el piano intemacional-- puede ser un fenomeno que alter© de manera susi 
que ya se venia practicando en terminos de transferencia de politicas. Roberto Rodriguez hace notar que:”Sobre la base de la 
la deuda externa que se verified en los primeros anos de la decada, acontecio un paulatino realineamiento de los paises de la | 
capital ista a los dictados de politica economica y financiera metropolitanos. Este proceso se concrete mediante el condiciona 
creditos estipulado por agencias tales como el Fondo Monetario Intemacional y el Banco Mondial - a traves de los Programs 
Ajuste Estructural - y tuvo como consecuencia la implantacion de proyectos restrictivos de la inversion publica y el gasto so< 
(1998:77 italicas del autor) 

Deuda extema-alineamiento-condicionamiento-organismos intemacionales parece una formula que hay que tomar en cuenta 
estudiar, en estos tiempos, quien influye sobre la educacion superior en Mexico y, sobre todo, a traves de que medios. Estos < 
deben tomar en cuenta tambien la nueva institucionalidad global en donde convergen y divergen diversos actores que crean r 
complejas de intercambio, conflict©, resistencia y acomodo. 

En resumen, el sistema de educacion superior en Mexico ha recibido historicamente influencia externa, aunque es claro que i 
procesos han ocurrido de acuerdo con variables endogenas como el papel de las elites politicas y economicas nacionales, y v; 
tipo exogeno tal como la crisis economica de principios de los ochenta, que modified el escenario para la participacidn, por e 
los organismos financieros intemacionales. 

Bajo este intrincado context©, es dificil pensar que en Mexico ha existido un modelo de reforma educativa propio y, ante la n 
actual, que se desarrollara un sistema educativo puramente mexicano cuando hay que enfrentar, como dice Marin, “las nueva 
realidades del entomo social mexicano y sus relaciones con los demas paises” (1998:13). 

B. Debate sobre la transferencia de poHtica educativa en Mdxico 

Aunque los procesos de transferencia de politicas publicas poseen, como se vio, diversas formas e involucran diversos adore 
debate, en terminos generales, se ha centrado en observar el papel que desempenan los organismos intemacionales (Note 1 1) 
agencias de financiamiento --especialmente, el Banco Mundial (BM) y el Fondo Monetario Intemacional (FMI)--. Esta secci 
identificar, mediante un analisis documental, que actores extemos influyen en el diseno de politicas educativas; a traves de qi 
mecanismos influyen; que puntos unen a los diversos organismos intemacionales y cuales son los impedimentos para transfe 
politica. Asimismo, se analizan los objetos que se podrian transferir asi como la forma en que se puede validar un proceso de 
transferencia de politicas. 

/. Agentes de la transferencia 

Para Alvaro Marin (1998), la globalizacion es una variable dependiente de las “potencias capitalistas” de Europa Occidental 
Norteamerica, por lo tanto, este fenomeno es promovido mediante “agencias intemacionales” como el Banco Mundial y el Fi 




Monetario Intemacional los cuales comenzaron a “disenar poHticas educativas mas expHcitas, en congruencia con las necesic 
supuestas o reales” de las empresas transnacionales (1998:25). Este autor califica los servicios que prestan las organizacione^ 
intemacionales como “peligrosas injerencias en la poUtica nacional” (1998: 19) y agrega que existen: intromisiones 

norteamericanas en muchas regiones y paises del mundo con el pretexto de modemizar la economia y actualizar los sistemas 
educativos para que respondan a los imperativos extemos...” (1998: 19). 

Marin interpreta los recientes cambios en materia educativa como una consecuencias de dominacion economica de las poteni 
hegemonicas como la estadounidense. Congruente con esta vision, Marin tambien resalta la participacion de la elites gobems 
dentro de estos procesos de “intromisiones”. En Mexico, anade este autor, “las autoridades educativas del sexenio anterior [1 
y principios del actual [1994-2000] sintieron la necesidad de impulsar una reforma a nivel superior que enfrentara la nueva n 
economica del pais y del mundo” (1998:13). Asimismo, puntualiza que “Las reformas educativas en Mexico se facilitaron pc 
excesiva dependencia de las instituciones universitarias de un presupuesto unico manejado por el gobiemo federal, asi como 
simpatia de los funcionarios e intelectuales formados en el exterior, quienes crearon un entomo amigable a la recepcion de ei 
propuestas.” (Marin, 1998:28). Segun Marin, “Se ha hecho creer a los dirigentes de los paises subordinados que es obligacioi 
respectivos Estados nacionales, formar ciudadanos como agentes economicos congruentes con la nueva organizacion social c 
a nivel mondial, bajo el liderazgo de los Estados Unidos como potencia hegemon ica (1998:20, cursivas nuestras). 

Con base en las opiniones de Marin, valdria la pena preguntarse si solo por sentir “simpatia” las autoridades mexicanas empr 
transformaciones educativas a finales de los anos noventa. ^Quien puede “hacer creer” a los dirigentes de algun pais que es 
“obligacion” formar individuos para la nueva realidad mondial? ^No tendran los gobemantes nacionales su propia racionalid 
puede 0 no estar acorde con los cambios a escala global? Los puntos de vista de Marin ignoran la forma en que realmente los 
de politica actuan y no hay oportunidad, bajo sus supuestos, de analizar los mecanismos de negociacion que pudiesen existir 
elites nacionales y las intemacionales (Note 12) . 

Armando Alcdntara analiza la forma de operar de las elites dentro de las tendencias mundiales de la educacion superior y die 
pesar de que los organismos intemacionales son potencias hegemdnicas que influyen en la politica educativa, es necesario to: 
cuenta que la reestructuracion de los sistema de educacion superior son una “transicion llena de adaptaciones, rechazos parci. 
conflictos” (2000:4). Ademas subraya que “conviene tener presente la existencia de esta mediaciones para evitar simplismos 
maniqueos para explicar la influencia de los organismos intemacionales y reducir la persistenciade las teorias conspiracionis 
anal is is comparado de la educacion superior (2000:14). 

El argumento de Alcantara es util para estudiar la forma en que los funcionarios de los organismos intemacionales interactua 
funcionarios de los paises pobres dentro de un intrincado sistema de relaciones e interacciones. Transferir, imponer, disemim 
por una parte, y adoptar, copiar o emular; por otra, es un proceso entre dos o mas agentes, el cual puede darse de una manera 
complejaque la que Marin observa. (Note 13) 

Por su parte, Pablo Latapi tambien reconoce que la influencia de organismos como el Banco Mondial en materia educativa s< 
otros, por una interaccidn entre expertos extemos y nacionales, pero que las politicas que el Banco propone “se negocian con 
en funcion de sus caracteristicas y los gobiemos mas fuertes pueden oponerse a algunos de sus elementos; por otra parte, en f 
implementacion intervendran las mediaciones y los actores nacionales que alteraran los resultados (Latapi, 1998:43) 

Es necesario resaltar que este experto en educacion, otorga un lugar a la negociacion entre funcionarios extranjeros y nacionc 
intercambio de ideas y opiniones entre estos actores es una parte poco documentada en el debate academico ya que la influen 
Banco no se ejerce de manera abierta sino que se lleva a cabo “en discretas negociaciones con los gobiemos” (Latapi, 1998:4 
forma de interactuar e influirdel personal de los organismos intemacionales con las elites locales -las cuales no solo incluye 
funcionarios sino investigadores, intelectuales, politicos, lideres sociales, rectores, en el caso de la educacion universitaria, y 
actores-- es un area que necesita mayor analisis para ampliar la comprension de la transferencia de politicas en Mexico. 

2. Medios de influencia: Coercion y convencimiento 

Tanto las mediaciones de las que habla Alcantara como las negociaciones privadas entre elites que observa Latapi conducen 
mas ampliamente sobre el papel que desempenan los organismos intemacionales y sobre sus medios de influencia yde persu 
este sentido, varios autores coinciden en senalar que el Banco Mondial presiona a traves de las condiciones de sus creditos pj 
adopten estrategias de desarrollo educative (Latapi, 1998). Tiinneimmann piensa que en los paises latinoamericanos las 
recomendaciones del Banco Mondial pueden influir la voluntad politica de los gobiemos con el proposito de no “arriesgar el 
los prestamos” (citado en Didriksson y Yarzabal, 1997:345). A su vez, Coraggio (1998) dice que el Banco Mondial “Tiene g 
que no se deriva de la claridad, la oportunidad, la precision teorica, metodoldgica o cientifica de sus ideas, sino del poder ecc 
que ejerce, en conjunto con el Fondo Monetario Intemacional, como palanca para imponer o sugerir con gran dureza las poli 
los gobiemos de America Latina.” (1998:50) 

Segun Coraggio, existen medios de coercion para presionar a los gobiemos nacionales para que adopten ciertas acciones, las 
cuentan con una base fundamentada y que otra funcion del poderoso organismos financiero tiende tambien a legitimar politic 
disenadas con anterioridad, pero i,quien o quienes son los autores de las politicas que el BM legitima? Este autor parece respi 
esta pregunta cuando escribe que [...] la teoria que esta por detras de las investigaciones elegidas por el BM para defender su: 
no es una teoria de la educacion. Es una teoria de la economia de la educacidn y, como teoria economica esta situada dentro < 
paradigma neoclasico que, durante este siglo, estuvo luchando por sobrevivir cuando muchas de sus falacias habian sido expi 
otras teorias.” (1998:52) 




Para Coraggio, promover las bondades del sistema economico “neoclasico” podn'a explicar la funcion de los organismos 
intemacionales como medios de legitimacion. Entonces el BM tiene dos funciones: influir en las agendas educativas naciona 
legitimar las poHticas disenadas previamente. Latapi tambien destaca la funcion legitimadora de otro organismo intemaciona 
Organizacion para la Cooperacion y el Desarrollo Economico (OCDE) que es “especialmente perceptivo de los cambios del : 
esta tentado tambien a convertirse en “instancia doctrinal” (2000b:59). 

En su caracteristico tono critico, el experto en educacion narra en su escrito lo sucedido en un seminario de la OCDE que sir 
marco para presentar el documento titulado Knowledge Management in the Learning Society: “El estudio de la OCDE y el s» 
no son inocuas discusiones academicas; forman parte de un proceso politico de legitimacion doctrinal, en este caso planetari( 
tendiente a imponer una determinada vision de la educacion (2000:60). 

Coraggio y Latapi coinciden en dos puntos muy importantes para el analisis de la transference de politicas. El primero es la : 
legitimadora del BM y de la OCDE, mientras que el segundo tiene que ver con la idea de que existe un actor m^ alia de las < 
institucionales que parece estar influyendo al sector educativo. Sobre este ultimo punto, Latapi agrega que “la OCDE se ha si 
Banco Mondial, al Fondo Monetario Intemacional y a otros organismos intemacionales en el proposito de conformar los sist* 
educativos y orientar la revolucion informatica de acuerdo con los intereses de las economias dominantes.” (2000:60) 

Para Latapi el “actor” que esta allende los organismos intemaciones son las “economias dominantes”, esto tiene cierta congn 
lo que dice Coraggio en el sentido de que es el sistema economico neoclasico quien esta “detras” de los organismos intemaci 
establecer esta relacion entre un espacio macroeconomico y un ambiente microinstitucional, Latapi y Coraggio conducen a p 
analisis el tercer tipo de transferencia de politicas: el coercitivo indirecto. Segun algunos autores, este tipo de transferencia 0 ( 
consecuencia de factores estructurales tales como las extemalidades, los cambios tecnologicos, la interdependencia economh 
desempeno actual de alguna nacion frente a otra (Dolowitz y Marsh, 1 996), y no existe, teoricamente, algiin agente determin: 
motivar alguna transferencia. Sin embargo, en la vision de Latapi y Coraggio, los organismos intemacionales son parte clave 
promover, legitimar o apoyar esos cambios estructurales y, muchas veces, lo hacen mediante formas coercitivas; asi que vale 
decir que un tipo de transferencia coercitiva indirecta puede convertirse en coercitiva directa o, quiza, ambas existan dentro c 
contexto de los paises en desarrollo. 

Otro punto que merece mayor atencion es que, aunque el BM y la OCDE son organismos intemacionales que puedan legitim 
politicas educativas a nivel “planetario” (usando el termino de Latapi); sus origenes, fines y medios difieren en cierta medida 
ejemplo, para John Lowe (Note 14) la OCDE sirve solo a cierto grupo de paises, mientras que el Banco lo hace a todo el mui 
1988). Ademas, la OCDE surgio en 1961 con el proposito de promover el bienestar economico de sus paises miembros que, • 
mayoria, son naciones relativamente ricas, mientras que el Banco Mondial nacio en 1945 y su funcion principal es ofrecer pr 
para incentivar el desarrollo economico, asi como ofrecer asistencia tecnica (Lowe, 1988). 

Para entender las formas de influencia de los organismos intemacionales dentro de la globalizacion, en general, y dentro de 1: 
perspectiva de la transferencia de politicas, en particular, es necesario conocer sus antecedentes y su estructura. La clasificaci 
organizaciones intemacionales presentada en el cuadro 2 podria ser un primer paso en este sentido. 



3. Institucionalidad de los organismos intemacionales. Convergencias y reacomodos 

Varios autores subrayan la convergencia que existe entre las politicas educativas de diversas organizaciones multilaterales cc 
Organizacion de Naciones Unidas para la Educacion, la Ciencia y la Cultura (UNESCO) o la OCDE y de los organismos finj 
intemacionales como el Banco Mundial o el Banco Interamericano de Desarrollo (vease Alcantara, 2000; Rodriguez y Alc^n 
2001). Segiin Alcantara, “al observar los desafios de la globalizacion” y los temas criticos del sistema de educacion superior, 
decir que “existe una coincidencia en muchas de las recomendaciones que tales organismos plantean para reestructurar la edi 
superior en una direccion que parece estar a tono con las crecientes necesidades del mercado y del Estado.” (2000:15) 



Pero, ^como se podrian explicar estas coincidencias? Segiin Lowe (1988), entre las organizaciones intemacionales no existe 
formal para armonizar los programas sino que existen algunos contactos que funcionan de manera informal. Este intercambic 
interpersonal podrian explicar estas convergencias. Ademas, dice Lowe, en la practica, las actividades de los organismos 
intemacionales no se duplican y casi nunca son “conflictivas” ya que, por un lado, la UNESCO y el Banco, estan m^ preocu 
lo que sucede en los paises menos desarrollados, mientras que, por el otro, la OCDE se ocupa de lo que hacen los paises con 
avance economico (Lowe, 1988:24). 

Asimismo, Rodriguez y Alcantara destacan que la posicion asumida por los bancos multilaterales en tiempos recientes denot; 
flexibilidad y mayor sensibilidad hacia las dimensiones politicas y sociales de cambio en materia educativa (2001). Como eji 
ello Alcantara apunta que en los recientes documentos publicados por el Banco Mundial sobre educacion superior “se deman 
participacion mas consistente del Estado, aunque se insiste en la importancia de que otros sectores de la sociedad tambien lo 
hagan” (2000: 11). Es decir, que instancias tanto piiblicas como privadas o civiles pueden participar en el diseno, gestion e irr 
de politicas sociales. Esto coincide con otros documentos del Banco como su reporte anual, World Development Report 1999 
el que presenta un Marco Integral de Desarrollo (CDF, por sus siglas en ingles) y lo propone como un medio para reducir la ] 
con “mayor efectividad”, con cuatro puntos que llaman la atencion, dado que se observa flexibilidad en la vision de este pod< 
organismo financiero: 

a). Los gobiemos necesitan “construir alianzas con el sector privado, las organizaciones no gubemamentales (ONG). 

las agencias de asistencia y las organizaciones de la sociedad civil para definir las necesidades de desarrollo e 



implementar programas” (1999:21). 



b) . Debe ser el pais y no las agendas de asistencia, el que tenga sus estrategias de desarrollo. 

c) . El Marco Integral de Desarrollo cuenta con una visidn de largo plazo y colectiva de las necesidades y soluciones, 
que deben formularse para lograr asi un “apoyo nacional sustentable” (1999:21). 

d) . “Las preocupaciones estructurales y sociales deberan ser tratadas de igual forma y al mismo tiempo que las 
preocupaciones financieras y macroeconomicas” (1999:21). 

Con su referenda a construir “alianzas con el sector privado, las ONG, las agencias de asistencia y las organizaciones de la s 
civil”, el Banco confirma la existencia de esa nueva institucionalidad que se menciono al principio de este articulo. Ademas, 
sus lideres empiezan a reconocer a los agentes locales y nacionales y ahora buscan que sus propuestas logren un “apoyo naci* 
sustentable”. Sin embargo, habrd que indagar tambien si esta nueva forma de actuar que del poderoso organ ismo finandero e 
congruente con sus principios enunciados en sus reportes y programas. 

4. Factores que restringen la transferencia de politicas en M^ico 

Como recordamos, Dolowitz y Marsh (1996) proponen seis factores que podrian facilitar o restringir el proceso de la transfei 
politicas (vease III-H). Estos factores son la complejidad de la politica, su aspecto adaptativo, las estructuras institucionales \ 
implementacion, la ideologia politica, la cultura y el lenguaje entre el emisor y el receptor de las politicas y las relaciones anl 
En seguida, se senalan algunos factores que limitan la influencia de los organismos intemacionales sobre la agenda educative 
Mexico. 

El Observatorio Ciudadano de la Educacion (Note 1 5) (OCE, 2001) responde a la nueva institucionalidad ya mencionada al t 
las prescripciones que el Banco realize en materia educativa. En su comuni cado titulado, Las recomendaciones educativas di 
Mundial para Mexico, el OCE advierte que el BM “ignora olimpicamente (sic) los estudios realizados por los investigadores 
mexicanos del pais y las propuestas formuladas por el magisterio y otros organismos de la sociedad civil” (2001 : Web-Site), ^ 
ademds, el analisis del organismo financiero sobre algunos puntos especificos carecen de una adecuada contextual izacion y n 
ponderan los factores politicos como es el caso del poderoso Sindicato Nacional de Trabajadores de la Educacion (SNTE) en 
aplicacion de las politicas recomendadas. Por esto, el Observatorio concluye que el andlisis del Banco fue “superficial” (OCf 
2001: Web-Site). 



La falta de vision para disenar e implementar politicas es congruente con la critica que hace Latapi Sarre (Note 16) (1998) so 
endeble base pedagogica en las que se sostienen las recomendaciones del Banco Mundial. En su escrito, Como educar sin pe. 
el experto mexicano en educacion senala que el modelo del Banco “nos regresa a la concepcion tradicional (transmisiva del 
aprendizaje), y apoya la ilusion de reformar la educacion sin atravesar por sus complejidades” (1998:45). Este autor tambien 
la falta de evidencia empirica de las prescripciones del Banco y de una “evaluacion seria” sobre estas, cuestionan la viabilida 
politicas (Latapi, 2000a). 



Por lo tanto, la descontextualizacion de las politicas sugeridas por el Banco, la omision de factores politicos, el reduccionism 
analisis sociales y la escasa base pedagogica y empirica son algunos factores que podrian impedir, hasta cierto grado, la trans 
de politicas del Banco hacia la realidad mexicana. El problema aqui es cuando existen mecanismos de coercion para transferi 
independientemente de la calidad de la politica que se quiera adoptar. Si se demuestra la debilidad en el diseno de politicas o 
programas, ^quien o como se podria impedir que se realice la transferencia? Alcantara sugiere que las instituciones universitj 
grupos de la sociedad civil, “exijan” a los negociadores de los acuerdos con los organismos intemacionales que informen pur 
verazmente de los avances, es decir, que haya transparencia en estos procesos (2000: 1 5). 

En caso de que se llevasen a cabo transferencias de politicas que son altamente cuestionables, los hacedores de politica nacio 
serian responsable directos de las consecuencias tanto politicas como economicas que ocurrieran y que podrian deslegitimar 
de cualquier gobiemo. Asi que discutir los asuntos piiblicos con grupos de expertos y con la sociedad civil, como sugiere Ale 
podria reducir los efectos regresivos que pudiera ocasionar una transferencia de politica. Es probable, desde nuestro punto de 
con la intencion de reducir estas consecuencias pudiese haber una mayor convergencia entre la racionalidad de los politicos c 
tomadores de decision y la de los investigadores educativos. 

5. Objetos de transferencia y complejidad para validar transferencias 

Es complicado identificar los objetos que se transfieren los organismos intemacionales hacia la agenda educativa mexicana. 1 
parte, Marin (1998) menciona que son “reformas educativas” las que el gobierno de Mexico ha adoptado como parte de la nu 
realidad global. Para Coraggio, lo que el Banco busca “imponer” o “sugerir con gran dureza” son politicas publicas (1998:50 
es mis especifico que Marin y Coraggio, y observa que el Banco pretende transferir a los paises pobres algunas politicas de 
financiamiento a la educacion (Latapi, 2000a). Sin embargo, este autor tambien piensa que la OCDE, mediante sus estudios ^ 
seminarios, puede diseminar ideas “tendientes a imponer una determinada visi6n de la educacion” (2000b). Por su parte, Ale. 
mas puntual cuando menciona “los temas criticos” que los organismos multilaterales han analizado en el caso de la educacioi 
Estos temas son el crecimiento de los sistemas, la pertinencia, la calidad, el financiamiento, el gobiemo y la eficacia, entre ot 
(2000:1 5). Este autor apunta que los organismos plantean “recomendaciones” para reestructurar la educacion superior y, “c c 
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no olvidar que el proceso de convertir las recomendaciones en politicas concretas pasa necesariamente por procesos mediacit 
los que entran en juego diversos actores a traves de resistencias, conflictos, negociaciones (piiblicas y privadas), etcetera.” (A 
2000:15) 

Es decir, se puede copiar una politica externa e incluirla en un programa o documento oficial, pero para mostrar esta transfen 
primero se tienen que comparar estos documentos y, una vez que se “presenten pruebas fisicas concretas, como sugieren Eva 
Davies, habra que analizar si esa politica se ha implementado o no para “determinar que tan lejos ha llegado la transferencia’' 
Davies, 1999:237-8). En este sentido, Alcantara piensa que “en ocasiones las recomendaciones requieren de largos periodos ] 
establecimiento” (2000:15). Asi que notar coincidencias, al comparar documentos oficiales con reportes extranjeros, solo es i 
para validar que existe un proceso de transferencia de politicas. Mostrar una evidencia mas solida del punto al que ha llegadc 
adopcion podria tomar tiempo si se espera que las politicas o programas, supuestamente transferidos, scan implementados. 



En el caso de la educacion superior en Mexico, es dificil precisar que objetos (tangibles e intangibles) se estan transfiriendo. 
Logicamente, es tambien complejo demostrar que alguna politica educativa se esta copiando o adoptando de algiin organisms 
intemacional. Latapi Sarre — quien ha fungido como asesor de varios secretarios de Educacion en Mexico— asegura que habr; 
“examinar si las propuestas del BM se estan reflejando en la politica educativa de Mexico, y discutir, en cada caso, si ello es 
conveniente o no” (1998:46). Esto abre interrogantes sobre la existencia real de procesos de transferencia de politica del BM 
Mexico. 



Sin embargo, Pablo Gonzalez Casanova da por hecho que existe una transferencia de politicas educativas de los organismos 
intemacionales hacia Mexico pues encuentra “extranas coincidencias” entre algunos documentos realizados por el Consejo 
Intemacional para el Desarrollo de la Educacion (Note 17) (CIDE, 1991) y un libro publicado por “altos funcionarios” de la ! 
de Educacion Publica (Gonzalez Casanova, 2001 :40). En el documento del CIDE, se hacian varias sugerencias que, segiin G- 
Casanova, “se convirtieron en la base de creencias, la argumentacion y la politica educativa que domina en el Mexico 
neoliberal” (2001 :40), y tambien nota semejanzas entre esta “base de creencias” y el documento de la OCDE titulado Exame. 
Politicas Nacionales de Educacion Superior OCDE (1997). (Note 1 8) Sin embargo, llama la atencion que en su “analisis 
documentado” (como el mismo lo llama), Gonzalez Casanova no cita algiin programa oficial antes de la publicacion del info: 
CIDE. Por eso valdria la pena preguntar si realmente el documento del CIDE fue “la base de creencias” para disenar e implei 
politicas educativas a nivel superior en Mexico. Para responder a este cuestionamiento, primero se seleccionaron cinco docui 
tres de ellos elaborados por autoridades mexicanas y dos por organismos intemacionales, todos ellos publicados en diferentej 
saber: El Programa Integral para el Desarrollo de la Educacion Superior (PROIDES, 1986) 

En seguida se compararon algunos elementos de diagnostico y acciones especificas que estos documentos incluian. Los resul 
esta sencilla labor se muestran en los cuadros 3 y 4. (Note 19) 

En el cuadro 3, se presentan los cinco programas seleccionados en orden cronologico y en las columnas siguientes algunos el 
de diagnostico incluidos en estos informes. El simbolo X significa que ese programa refiere en su texto, la expansion masiva. 
planeacion y coordinacion, y la desercion de alumnos en las universidades mexicanas. Siguiendo este orden, puede observars 
tanto los tres programas gubemamentales como los dos informes de los organismos intemacionales coinciden en senalar que 
principal del detrimento de la educacion superior fue su anarquico crecimiento, y excepto la OCDE asegura que la expansibn 
desordenada del sistema de educacion superior en Mexico imposibilito una planeacion adecuada. 



Cuadro 3. Algunos elementos de diagnostico en la educacibn superior en Mexico 





Problemas derivados de la expansion masiva en las 
universidades 


Falta de planeacion y coordinacion en 
el sector 


de 


PROIDES 

1986 


X 


X 




PME 1989 


X 


X 




CIDE 1991 


X 


X 




PDE 1995 


X 


X 




OCDE 1997 


X 







Puente: Elaborado por el primer autor, con base en los programas de referencia. 

Otro elemento en comiin que mencionan los dos organismos intemacionales (CIDE, 1991 y OCDE, 1997), as! como el PROI 
desarrollado en 1986, y el PDE de 1995 es la desercion escolar la cual, segiin, los organismos intemacionales es una consecu 
las desventajas economicas de los jovenes mexicanos. El cuadro 3 tambien expone que, antes de la publicacion del documeni 
CIDE, las autoridades mexicanas ya habian identificado algunos de los principals problemas del sistema de la educacion su] 
lo tanto, los documentos del CIDE y de la OCDE no aportaron elementos que los hacedores de politica mexicanos no conoci* 
previamente. Estos resultados motivan a profundizar en el analisis y comparar algunos objetivos y estrategias de la educacioi 
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En el cuadro 4, se exhiben algunas poHticas que las autoridades mexicanas habian planteado para reformar la educacion supe 
Como puede observarse, la evaluacion de estudiantes, profesores y procesos educativos, la aplicacion de examenes de selecc: 
ingresar a la educacion superior, la diversificacion de fuentes de financiamiento, y la vinculacion con el sector productivo fut 
estrategias planteadas por el gobiemo mexicano antes de que los dos organismos intemacionales seleccionados (el CIDE y la 
desarrollaran sus informes. 



Cuadro 4. Algunas politicas de educacion superior en Mexico 





Evaluacion de 
estudiantes, profesores y 
procesos educativos 


Examenes de seleccion 
para el ingreso a la 
educacion superior 


Diversificacion de las fuentes 
de financiamiento en las 
universidades 


Vinculacion d 
con el apa 
productivo d 


PROIDES, 

1986 


X 


X 


X 


X 


PME, 1989 


X 


X 


X 


X 


CIDE, 1991 


X 


X 


X 


X 


PDE, 1995 


X 


X 


X 


X 


OCDE, 

1997 


X 


X 


X 


X 



Fuente: Elaborado por el primer autor, con base en los programas de referenda. 

En su informe la OCDE otorga un reconocimiento a la politica educativa mexicana y senala que, a pesar de que Mexico puec 
beneficiarse de la experiencia del resto de los paises que forman esta organizacion, tambien “puede ensenar mucho en termin 
soluciones desarrolladas por el mismo” (1997:223). Es decir, este pais puede no solo podria adoptar politicas o programas de 
extranjero sino tambien transferirlos. A pesar de esto, habria que ir mas alia del discurso y analizar que trayectoria podrian se 
procesos en el caso de los paises en desarrollo. El esquema de trayectorias propuesto por Evans y Davies podria ser de utilidi 
sentido (vease el cuadro 1). 

Asimismo, destaca que algunos de los programas revisados se citan y se critican entre ellos mismos. Por ejemplo, el PME re< 
se tomaron las recomendaciones del PROIDES para ’’plantear” las lineas esenciales para la modemizacion de la educacion si 
mexicana. Mientras que el CIDE se aventura a decir que si las estrategias del PROIDES “se hubieran implantado con rapidc 2 
constituido un paso para iniciar la mejora de la calidad” (CIDE, 1991:40). De la misma manera, el informe del CIDE critica j 
debido a que lo encuentra ’’’excesivamente” vago, ya que no senala a quien le corresponde hacer que. Como se observa, esta 
critica del informe del CIDE resalta diversos aspectos sobre la implementacion de politicas publicas, por lo que cabria tambi« 
posibilidad de que este tipo de estudios motiven a los diversos actores locales a poner en practica sus politicas educativas. 



A1 contrastar algunas acciones contenidas en los programas e informes aqui seleccionados, parece excesivo opinar que las su 
contenidas en el informe del CIDE fueron “el punto de referencia ineludible” para el diseno de politicas en educacion superic 
(Gonzalez Casanova, 2001: 37). Las coincidencias entre el material bibliograflco desarrollado por las autoridades mexicanas 
la publicacion de los informes del CIDE y la OCDE sustentan esta aseveracion; sin embargo, hay vetas de analisis que se abr 
direccidn, como: ^Hasta que grado ha llegado la transferencia de politica educativa a nivel superior en Mexico? Si se toman < 
el esquema de TP descrito anteriormente y los argumentos del debate academico, podria decirse que la transferencia de politi 
educativa que ha ocurrido en Mexico ha sido solo de tipo coercitivo y que se ha producido por medio de algunos organismos 
intemacionales. No obstante, tambien en el presente trabajo se ha afirmado que existen negociaciones privadas y publicas en- 
gobiemos receptores y alguna organizacion que desea “imponer” una politica publica. Ademas, la influencia que llegaran a U 
organismos intemacionales sobre las agendas nacionales seria de acuerdo con la naturaleza, funciones, origenes y estructura * 
organismos, es decir, si algunos utilizan mecanismos de coercion para “imponer” una politica, otros legitiman y, quiza, tamb 
presionan, por otros medios para que se adopten politicas, programas o instituciones. En todo este proceso la interaccion entr 
tambien juegan un papel destacado. 



Asimismo, como se anoto en parrafos anteriores, existe una reformulacion del papel de los organismos intemacionales, come 
Mundial, que tiende a tener mayor flexibilidad y sensibilidad en los cambios que desea dirigir (Rodriguez y Alcantara, 2001) 
habrd que investigar si los renovados propositos de este organismo pueden tener un efecto en la politica educativa de los pais 
desarrollo o si forman parte de un movimiento estrategico para redimirse de las criticas que ha recibido por su forma de open 
20 ) 



A traves del debate academico surgido en Mexico, es dificil identificar que objeto u objetos se estdn transfiriendo. Por un lad 
de reformas universitarias y, por otro, de ideas y politicas concretas. Esto dificulta la validacion de los procesos de transferen 
politica educativa, ya que tampoco se han desarrollado herramientas metodologicas para demostrar estos procesos. Asegurar 
gobiemo mexicano ha copiado del CIDE programas o politicas en su totalidad es impreciso ya que, en este caso, existen coin 
entre los programas de educacion superior mexicanos, que fueron elaborados antes que el informe del Consejo. Ademas la O 
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(1997) reconocio que Mexico puede ensenar mucho en terminos de politica educativa, por lo que cabn'a la posibilidad de que 
tambien realizara transferencias hacia otros contextos. 

Conclusiones 

La globalizacion motiva y facilita la adopcion de politicas, programas, ideas o instituciones de un contexto a otro. La politica 
educacion superior en Mexico se ha adaptado a las circunstancias de su cambiante entomo e historicamente ha recibido la ini 
mod el os educativos extranjeros, aunque cada etapa ha tenido variables intemas o extemas que han determinado significativai 
procesos de transferencia de politica. A lo largo de la historia, se han generado profundos cambios en las funciones del gobie 
tampoco se puede soslayar el surgimiento de una institucionalidad global cada vez mas compleja, con altos grados de interac 
comunicacion entre multiples actores, lo que produce un ambiente idoneo para que se adopten politicas desarrolladas en otro; 
ambientes. 

A1 relacionar las principales lineas de debate sobre la adopcion de politicas educativas a nivel superior en Mexico con el esqi 
transferencia de politicas, se observa que este analisis se ha centrado primordialmente en el papel que desempenan los organi 
intemacionales en la transferencia de politicas y en los medios de presion que utilizan para influir la agenda educativa nacion 
embargo, aun quedan por investigar diversos aspectos para tener una nocion mas precisa de la transferencia de politicas educ 
como de la amplia gama de agentes que podrian influir de manera muy diversa en el diseno de las mismas. Estudiar a los div 
agentes de las transferencias, asi como conocer sus fines, objetivos e intereses, ayudaria a definir que tipo de transferencia (v 
coercitiva) ocurriria. Ademas, podria explicarse la forma en que se combinarian los mecanismos de influencia o presion de c; 
en este proceso. Tambien, es necesario tener en cuenta que las transferencias de politicas podrian utilizarse como una herram 
para disenar acciones encaminadas a la solucion de problemas, en este caso, educativos, y de esta manera, eventualmente, ah 
tiempo y recursos. (Kote 21) 

Si se toma en cuenta que las transferencias deben ser un proceso consciente, independientemente de su tipo, como sugieren E 
Davies (1999); entonces, se asume que puede haber negociaciones, resistencias y, hasta conflictos, entre la parte emisora y la 
de politicas publicas, por lo que es necesario investigar la forma en que los posibles agentes de la transferencia interactuan er 
ambiente institucional cada vez mas complejo. 

A partir del analisis elaborado, se concluye que es dificil identificar los objetos tangibles e intangibles que realmente se han t 
a la agenda educativa mexicana. A1 carecer de este conocimiento previo, es logico que validar la existencia de la adopcion dc 
politica educativa en Mexico sea un ejercicio complicado y abierto a multiples cuestionamientos. 

Los argumentos que sugieren que Mexico copia politica educativa a nivel superior de algunas organ izaci ones carecen de sust 
las autoridades mexicanas ya habian planteado, con varios anos de anterioridad, acciones muy similares a las que el CIDE y . 
hicieran en su momento para resolver los problemas de la educacion universitaria en Mexico. 

A1 no haber, por un lado, una clara evidencia de que Mexico copia politica educativa de algunos organismos intemacionales • 
determinado periodo y, por otro, existen coincidencias entre programas nacionales e informes intemacionales, surgen, entonc 
algunos cuestionamientos: i,por que existen estas similitudes?, y ^como se producen? Si la respuesta fuera que son las macro* 
(globalizacion, sistemas economicos, etc.) las que dictan estos cambios y los hacedores de politica los que aprenden de estos; 
asumirse que seria posible que se hubiese dado un ejercicio consciente y voluntario para copiar o emular una politica. Sin err 
entonces, iqut papel desempenan los organismos intemacionales? Los multiples mecanismos de presidn e influencia de estaj 
organizaciones son otra parte muy importante, aunque no exclusiva, para copiar o emular determinada politica publica dentrc 
ambiente de creciente complejidad, por eso necesario el analisis sistematico que ayude a mejorar y a ampliar el conocimientc 
transferencias dc politica publicas en Mexico. 
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Notas 

1. Para un estudio mas amplio sobre la global izacion, vease Lanni (1999). 

2. Este teimino es una traduccion del teimino “hollowing state” usado por R.A.W Rhodes (1997). 

3. Hasta el momento no conocemos una traduccion en espanol del teimino “governance” que sea conocida de manera genera 
que en este trabajo usamos “si sterna gubemamental” como sinonimo de ''governance''. 

4. Para un analisis intemacional de las redes de politicas publicas, vease Evans (1998). 

5. Entrevista de Andres Ortega en el diario espanol El Pais, 25 de julio de 1999. 

6. Los terminos “resistencia” y “acomodo” pertenecen a Cemy (1999). 

7. En este sentido, la confusion es mayor cuando Gonzalez Casanova habla de que la nueva reforma universitaria sigue “estri 
una “logica modemizadora que es, a la vez, neoliberal y neoconservadora” (2001:107). Coraggio (1998) tambien usa indistin 
tres terminos: “paradigma neoliberal” “paradigna neoclasico” y “propuesta” o “decision ideologica neoconservadora”. Castrc 
apunta que existe una corriente de pensamiento llamada “neoliberalismo, o paradojicamente, neoconservadurismo” (1997:18 



8. Phillip Brown (1999) usa el termino de “importacion de politica” para analizar temas educativos como la formacion de hal 
dentro de las corrientes econdmicas actuales. 



9. Garciadiego, citado en Gil (2000:24). 



10. Sobre la Reforma Universitaria de Cordoba, vease Tunnermmann 1997. 



11. Para un analisis descriptivo de las organizaciones intemacionales sobre el ambito educativo, vease Stephens (1988). 



12. Para el caso latinoamericano, Puiggros parece coincidir con la vision de Marin cuando dice que los presidentes de paises 
sudamericanos como, Pinochet, Menem, Fujimori, Sanguinetti y luego Cardoso recibieron “las orientaciones politico-educati 
paquete programatico del Banco Mundial sin realizar adaptaciones a la realidad de su pais y sin condicionar ningun aspecto ( 
reforma exigida” (Puiggros, 1999:7). 

13. Flores-Crespo (2001) describe una forma en que algunos especialistas de la educacion superior del Banco Mundial intera 
las comunidades de academicos y funcionarios de diversas naciones. 

14. John Lowe (1988) hace un interesante estudio de la estructura y las funciones de cuatro organizaciones intemacionales (E 
Europeo, la OCDE, la UNESCO y el Banco Mundial) en el campo de la educacion. Otro no menos importante analisis sobre 
organismos intemacionales y educacion superior es el de Allende et al, (1998, Mexico: ANUIES). 



15. El OCE es una asociacion civil formada por investigadores, maestros, lideres de opinion, legisladores, estudiantes, etcete 
proposito de examinar el diseno e implementacion de la politica educativa en Mexico asi como de fomentar la critica indeper 
la participacidn de las autoridades en la discusion de los problemas educativos. El Observatorio publica quincenalmente sus 
comunicados en el periodico La Jornada. Para mayor informacion vease www.observatorio.org 



16. Latapi tambien ha comentado de manera critica otros tantos documentos de organismos intemacionales. Por ejemplo, ves 
informe Delors” (1997), “La resena de la OCDE sobre educacidn superior” (1997) o “PDNU: el cambio es posible” (2000a). 

17. El CIDE fue fundado en 1970 como una institucion independiente, sin fines de lucro y con le objetivo de realizar estudio: 
politica comparada. Tambien habriaque recordar que el informe del Consejo fue coordinado por Phillip Coombs quien fungi 
asesor “extemo” de un ex secretario de educacion en Mexico durante el sexenio 1988-1994. 



1 8. Gonzalez Casanova tambien cita documentos elaborados por algunos economistas ligados al Banco Mundial que guardar 
correspondencia con la propuesta del ex rector de la UNAM, Francisco Barnes de aumentar las cuotas de colegiaturas a los e 
y sus familias. Esta propuesta se puso en marcha en 1999 lo que produjo un paro estudiantil de casi diez meses en una de las 
grandes universidades latinoamericanas. 

19. Estos resultados forman parte de la tesis de maestria elaborada por el primer autor (vease Flores-Crespo, 1998). 
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20. Para un analisis a fondo sobre el papel del Banco Mondial dentro del ambito de la educacidn, vease Jones (1992). 

21 . Un caso de una transferencia voluntaria en Mexico fue la creacion de las universidades tecnoldgicas (UT) en 1991 que er 
modelo educativo de los institutos tecnoldgicos universitarios (lUT) Franceses creados a mediados de los anos sesenta. Para i 
comparacion del desempeno de ambas modalidades educativas, vease Villa Lever y Flores-Crespo (2002). 
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Abstract 

There is a significant negative relationship between the percentage of teachers on emergency 
permits and student achievement at the school level in California schools, after controlling for 
other student and school characteristics. Generally, the more emergency permit teachers there are 
in a school, the lower the school's achievement. This phenomenon is examined in the context of 
other contributors to student achievement such as socio-economic status and school size. The 
effects of teacher distribution and school selection as contributing factors are considered. In 
addition, policy and legislative initiatives related to emergency permit teachers that have been 
recently debated in California will be discussed. Finally, a set of initiatives is proposed that 
attempt to decrease the need for emergency permit teachers and ensure that those that must be 
hired due to shortage conditions have the support they need to become credentialed teachers. 



Introduction 

Class size reduction, teacher retirement and attrition, and a burgeoning school-age population have all played roles in 
recent severe shortages of qualified K-12 teachers throughout California (EdSource, 2001). One of the consequences of 
shortages of qualified personnel is that large numbers of teachers have been hired on emergency permits. In the 2000- 
01, 34% of all first year teachers in California were emergency permit (EP) teachers, and 10% of all California teachers 
held emergency permits (Note 1). As will be documented in this paper, EP teachers tend to be concentrated in schools 
with low standardized test scores, high percentages of minority students and English learners, and high percentages of 
students with free or reduced-price lunch status. 

Because of the correlation between high percentages of EP teachers in schools and low student test scores on 
standardized assessments such as the SAT-9, California legislators proposed several pieces of legislation in 2001 
designed to limit the number of EP teachers in schools and districts. The proposed bills varied from an outright cap on 
the percentage of EP teachers in a district to complex formulas for determining the “teacher quality” in a given school 
and district accompanied by a plan to improve the quality of the teaching force. All of the proposals were designed to 
decrease the number of EP teachers employed by districts, and/or redistribute EP teachers where they are concentrated 
in low-performing schools. 

In this paper, I will 1) examine the current distribution of emergency permit (EP) teachers; 2) discuss the association 
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between EP teachers and low test scores; 3) consider current legislative solutions to the EP teacher “problem” in 
California; 4) discuss how the policy system can influence teacher distribution; 5) discuss the preparation available to 
teachers who are currently entering teaching on emergency permits 

Teacher Credentials and Quality 

Emergency permit teachers, California has been hard-pressed in recent years to find enough fully credentialed teachers 
to fill its classrooms. Shortages of qualified teachers are not spread across all schools and districts equally, and some 
schools and districts suffer more severely from hiring difficulties than others. Urban districts in particular have had 
difficulty recruiting sufficient numbers of qualified teachers. The problem of recruiting teachers to work in urban 
schools is not confined to California, and urban areas in many states have faced similar shortages (Fideler, Foster, & 
Schwartz, 2000). Urban schools frequently hire EP teachers to fill their staffing needs after exhausting other teacher 
hiring mechanisms (Note 2). 



Teacher quality. Since legislation passed in 1990 making it possible for teachers to begin K- 12 teaching careers 
without full credentials, there has been ongoing concern about the quality of EP teachers. Studies in California have 
suggested that the students most in need of qualified, highly trained teachers are least likely to get them (Center for the 
Future of Teaching and Learning, 2001; Public Policy Institute of California, 2000; Shields et al., 2001). The students 
generally considered most in need of highly qualified teachers are those attending schools where student standardized 
test scores are low and where there are large percentages of low-income students and/or minority students and English 
learners. 



Teacher quality and student achievement. As shown in Figure 1 below the percentage of EP Teachers increases 
monotonically as student achievement (using school API scores (Note 3)) decreases. Thus, the top-performing schools 
in the state (Decile 10), have about 5% EP teachers on average, though many schools in this decile have no EP 
teachers. Conversely, schools in the lowest performing schools in the state (Decile 1) have an average of about 23% EP 
teachers, though there are some schools with more than half the faculty teaching on emergency permits. It should be 
emphasized that the high correlation between having many EP teachers and having low test scores demonstrates an 
association, but does not necessarily suggest a causal relationship. 



Distribution of EP Teachers by State Ra nk 
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Figure 1. Relationship of CAP Performance Index and Percentage of EP Teachers 

Data Source: California Department of Education (see Appendix A) 

Note: Decile 1 is /owe^z-performing. Decile 10 is highest-performing 

Figure 1 demonstrates the relationship between percentages of EP teachers and decile rankings on California’s 
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Academic Performance Index. Data for EP teachers and state rankings are based on school year 1999-2000. 

Research has shown that teaching has an important and substantial impact on student achievement. A study using 
Texas data matching student gain scores to teachers demonstrated that individual teachers have a much stronger 
influence on student achievement than previously suspected (Hanushek, 1998). But does teacher credentialing matter? 
Many studies suggest that uncredentialed teachers are less effective in the classroom than teachers with credentials as 
measured by student achievement. (Darling-Hammond, 1994; Darling-Hammond, 2000; Fetler, 1999; Fuller, 2000; 
Goldhaber & Brewer, 2000; Hawk, Coble, & Swanson, 1985). While it is difficult to prove a causal connection 
between EP teachers and poor student achievement, (Note 4) there is strong and substantial evidence of a correlation 
between these variables, even after other potential sources of influence, such as socioeconomic status of students and 
other school resources, are taken into account. 

Some researchers contend that there is insufficient empirical evidence to claim that having teacher certification results 
in better teaching and higher student achievement (Ballou & Podgursky, 1999; Goldhaber & Brewer, 1996, 2000, 

2001 ; Hanushek, 1994). The lack of consensus among education researchers is an indication of how difficult it is for 
observational research to establish connections between particular teacher qualities and qualifications (such as 
credentials) and particular student outcomes (such as test scores). 

Even if it is possible to connect student achievement with particular teachers, it is more difficult to establish which 
particular practices, strategies, skills, knowledge, communication ability, etc. contributed to the students' achievement, 
whether good or bad. Further, the school conditions and context in which teachers work may have a substantial impact 
on their ability to teach effectively. And since many EP teachers work in schools where facilities are inadequate, 
teaching resources are scarce, and teaching conditions are difficult, it is hard to determine what percentage of a 
students' test scores are related to teacher credentialing and what percentage are related to extraneous factors in the 
classroom or school. 



Researchers who are concerned about the quality of EP teachers realize that more research needs to be done in this 
area. The question to be considered in looking closely at EP teachers' impact on student achievement is ""How does 
certification matter?” As Darling-Hammond has pointed out, it is crucial to determine what certification actually means 
in terms of different qualities and performance among teachers (Darling-Hammond, Berry, & Thoreson, 2001). 

Some states (Tennessee, for example) have developed or are developing student-level record-keeping systems that 
should allow researchers to better examine the effects of teachers on students (Anderson, 1998). Texas has also made 
matched student and teacher data available to interested researchers. With such data, student achievement information 
can then be further examined with reference to the certification of the teachers. Unfortunately, California is years away 
from implementing a student-level tracking system that will allow individual student achievement to be tracked by 
teacher and school. Thus, further exploration of the association between credentials and student achievement is still a 
long way off for California. 



Teacher qualifications 



States vary widely in the qualifications they demand of teachers. California relies heavily on a single test, the CBEST 
(California Basic Education Skills Test) as the initial gateway into teaching,. Credentialed teachers must also pass 
subject matter tests or complete a subject matter approved program, a test of reading knowledge, and complete other 
teacher education requirements, but these requirements do not pertain to the nearly 40,000 teachers on emergency 
permits or waivers. California places less importance on graduation from an approved preparation program or having a 
teacher credential than almost any other state, ranking second-to-last in the nation (just ahead of Florida) in the 
percentage of districts demanding these qualifications (46.4%) (National Center for Education Statistics, 2002). 
California is moving towards performance evaluation and induction as requirements for teacher credentialing, but little 
evidence exists about how induction benefits new teachers, including those without preservice preparation, or how 
performance evaluation helps ensure better teaching. 

There are many factors that contribute to student achievement that cannot be measured by whether or not teachers have 
clear credentials. Consider that there are many schools in California that are “underperforming” (Note 5) even though 
they do not have high percentages of EP teachers. An analysis of API scores and percentages of EP teachers in schools 
reveals that 15% of California schools that were “underperforming” in 1999-00 had below the median number of EP 
teachers. There were more than 200 schools that scored in the lowest two deciles of the API (considered seriously 
underperforming) yet had fewer than 8% EP teachers. And there were 60 schools in the lowest two deciles with 1% or 
fewer EP teachers. These findings illustrate the importance of considering teacher characteristics and qualifications 
besides teaching credentials that may contribute to student achievement. 

Teacher education and experience 

California's EP teachers have generally completed a BA or BS degree and passed the CBEST (California Basic 
Education Skills Test), but have not completed teacher preparation coursework or passed the required content-area tests 
for teachers in California. While some EP teachers are transfers from out-of-state who teach on an emergency permit 
while completing California requirements, most EP teachers are novices who have had no teacher training and who 
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have never taught in a K-1 2 public school. Many are recruited from teacher education programs before they have 
finished their coursework, a practice which was rampant in California after class size reductions in the 1996-97 school 
year and growth in the student population in California resulted in some districts desperately scrambling for teachers. 

EP teachers not only lack specific teacher preparation coursework, they are also less likely to hold higher degrees such 
as masters’ degrees (Note 6). The evidence about whether teachers’ overall education levels are correlated with student 
test scores is mixed. For example, while some studies on teachers’ education levels have shown that having a master’s 
degree has little effect student achievement (Ferguson & Ladd, 1996; Monk, 1994; Wenglinsky, 2002), other studies 
have found small effects of teachers’ education levels on student achievement (Ferguson, 1991; Public Policy Institute 
of California, 2000).Teaching experience also has been found to have variable effects on student achievement. Some 
studies find little effect (Monk, 1994; Wenglinsky, 2002), while others suggest that experience has a small overall 
effect on student achievement (Ferguson & Ladd, 1 996; Sutton & Soderstrom, 1 999). Other researchers have found 
that lack of teaching experience appears to have a negative impact in the first few years of teaching (Betts, Rueben, & 
Danenberg, 2000), but that there is not a clear linear relationship between teaching experience and student achievement 
(Hanushek, 1998). Thus, while teaching experience and higher levels of education are valuable to schools and 
classrooms for many reasons (Note 7), the evidence about how these characteristics impact student performance is 
inconsistent. 



Subject-specific training may also be important to student test scores. Several studies have found that having subject- 
specific training has a significant impact on secondary math and science achievement (Fuller, 2000; Goldhaber & 
Brewer, 1996; Monk, 1994; Wenglinsky, 2002). Thus, ensuring that the teachers hired have an appropriate major or 
minor for the subject they are teaching may be at least as important to student test scores as previous teaching 
experience or advanced degrees. This is an important consideration in designing a plan for increasing student 
achievement through higher quality teaching. 



Pedagogical coursework has also been found to be associated with higher student achievement. Wenglinsky used 
multilevel structural equation modeling on NAEP data and found that some teacher inputs are positively correlated 
with student achievement (Wenglinsky, 2002). He found that particular professional development topics (higher-order 
thinking skills and methods of teaching diverse learners) were positively related to student achievement, as were 
specific classroom practices (hands-on learning and employing higher-order thinking skills). This study could be 
construed as evidence that at least some pedagogical training matters, though it is not clear whether teacher preparation 
programs or professional development are better delivery systems for this training (Wenglinsky's measure included 
both college coursework and in-service training). Similarly, Monk found that pedagogical training in subject matter 
methods was positively correlated with student achievement in math and science, sometimes even more strongly than 
subject matter knowledge (Monk, 1 994). 



Ethnicity of California’s EP teachers 



As shown in Table 1, EP teachers in California are considerably more likely to be from a minority group than are fully 
credentialed teachers. They are more than twice as likely to be Hispanic or Latino (26.1% vs. 10.7%), more than twice 
as likely to be African American (10.5% vs. 4.8%), and more likely to be Asian (4.4% vs. 3.9%), Pacific Islander (.3% 
vs. .2%), and Filipino (1.4% vs. .9%). EP teachers are slightly less likely to be American Indian (.6% vs. .7%), and 
considerably less likely to be white (55% vs. 77.9%). 



Table 1 


Characteristics of California EP vs. Credentialed Teachers 




EP Teachers 


Credentialed Teachers 


% Doctorate 


.9 


1.7 


% Master’s + 30 or more hours 


4.7 


20.1 


% Master’s degree 


7.9 


17.5 


% Bachelor’s + 30 or more hours 


27.2 


46.0 


% Bachelor’s degree only 


58.4 


13.7 


% Less than Bachelor’s degree 


.7 


.6 


Years of service in district 


2.46 


11.72 


Years of teaching 


3.25 


14.70 


% Male 


34.6 


28.1 


% Female 


65.3 


71.7 
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% American Indian/Alaska Native 


.6 


.7 


% Asian 


4.4 


3.9 


% Pacific Islander 


.3 


.2 


% Filipino 


1.4 


.9 


% Hispanic or Latino 


26.1 


10.7 


% African American, Not Hispanic 


10.5 


4.8 


% White, Not Hispanic 


55.0 


77.9 


% Multiple Race or No Response 


1.6 


.9 


% Authorized to teach English 


8.1 


13.2 


% Authorized to teach Life Sciences 


4.7 


4.8 


% Authorized to teach Mathematics 


6.9 


6.2 


% Authorized to teach Physical Science 


3.7 


3.3 


% Authorized to teach Special Education 


15.7 


12.8 


% Authorized Reading Specialist 


.5 


2.2 


% Authorized Bilingual Teaching 


4.3 


10.1 



Data Source: California Department of Education (see Appendix A) 

The ethnic composition of California's schools is becoming increasingly more minority and less white. Thus, the 
greater number of minority teachers entering the profession through alternative pathways than through traditional 
programs may be beneficial to California in terms of moving towards a teaching force that is more representative of the 
students being taught. This is an important consideration in planning ways to decrease the numbers of emergency 
permits while increasing diversity among the teaching force. Further, it is interesting to note that, unlike white teachers, 
African-American and Hispanic teachers are less likely to transfer away from schools with high percentages of 
minority students and more likely to transfer into schools with even higher percentages of students of the same ethnic 
backgrounds as themselves, regardless of the students' poverty or achievement (Note 8) (Hanushek, Kain, & Rivkin, 
2001). It could be argued that recruiting more minority teachers into the teaching force could result in the creation of 
more stable teaching staffs at schools with high percentages of African-American or Hispanic students. 

Teacher Distribution 

Grade levels taught by EP teachers. EP teachers are most likely to be hired by a middle school or junior high. One 
reason for this is that teachers are most likely to transfer away from, rather than into a middle school (Chester, 
Offenberg, & Xu, 2001). These vacancies are then disproportionately filled with either new, recently credentialed 
teachers, or with teachers holding emergency permits. Second to middle schools, EP teachers are most likely to work in 
high schools. 

The distribution of teachers nationally. 

Data from the National Center for Education Statistics provides some insight into patterns of hiring by school 
characteristics (NCES, 1996). The data shows that newly hired teachers in urban schools are more likely to be 
beginning teachers, and newly hired teachers in urban fringe/large towns (i.e., the suburbs), are more likely to be 
transfers. Even more striking is the breakdown of hiring patterns by student socioeconomic status. The data shows that 
newly hired teachers in high poverty central city schools arc far more likely to be beginning teachers than transfers. In 
addition, newly hired teachers at schools with 20% or more minority students are considerably more likely to be 
beginning teachers than transfers (Table 2). 



Table 2 






National Distribution of Newly Hired Teachers (NCES Data) 




School and Student Characteristics 


Beginning 

Teachers 


Transfers 


Hired in Central City 


43.2% 


27.7% 
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Hired in Urban Fringe/Large Town 


39.6% 


34.2% 


Hired in Central City school with 41-100% free/reduced price lunch students 


45.8% 


25.6% 


Hired in Urban Fringe/Large Town with 0-5% free/reduced price lunch students 


36.2% 


42.3% 


Hired in Urban Fringe/Large Town with 41-100% free/reduced price lunch 
students 


52.2% 


25.7% 


Hired in Rural/Small Town with 41-100% free/reduced price lunch students 


56.0 


28.5% 


Hired in schools with less than 20% minority students 


42.8% 


35.7% 


Hired in schools with 20% or more minority students 


46.2% 


29.0% 



Source: NCES (1996) Sources of Supply of Newly Hired Teachers 



This national data can be construed as confirmation of what district level researchers are finding about teacher 
transfers, i.e., that they are transferring away from high-poverty, high-minority schools into schools with lower levels 
of poverty and fewer minority students (Chester et al., 2001). Research done at the state level in New York reveals a 
similar trend, with teachers moving away from schools with mostly high poverty, low-achieving students and into 
schools with fewer minorities, less poverty, and better achievement (Lankford, Loeb, & Wyckoff, 2001). Some of this 
effect appears to be due to the fact working conditions and salaries are frequently also lower in these schools. In 
California, recent analyses suggest that these factors matter even more than student characteristics in predicting high 
levels of teacher turnover (Loeb, Darling-Hammond, & Luczak, 2002). 

The distribution of EP teachers across the state of California 

The statewide average of EP teachers per school was 1 1 .5% (Note 9) in 1999-2000. But in urban schools, the average 
percentage of EP teachers climbed to 14.6% for the same time period, compared to less than 7% in small towns and 
rural areas. Further details are shown in Table 3. This provides clear evidence that more EP teachers are finding 
employment in urban school districts, which are likely to be lower in student achievement and socio-economic status 
and higher in the percentage of minority students and English language learners. Table 4 divides the percentage of EP 
teachers at schools into two halves at the median. This table confirms that schools with higher percentages of EP 
teachers are also likely to have nearly twice the percentages of African American and Hispanic students, and half the 
percentage of white students. Similarly, much higher percentages of EP teachers are found in schools with low income 
students. And in schools with higher percentages of EP teachers, there are nearly twice as many English language 
learners. 



Table 3 


Distribution of California EP Teachers by Population 


Population Status 


Mean Percentage of EP Teachers 


N (schools) 


Large City (pop > 250K) 


14.61 


1427 


Mid-Size City (pop < 250K) 


9.01 


973 


Urban Fringe of Large City 


11.81 


3210 


Urban Fringe of Mid-Size City 


9.47 


425 


Large Town (pop > 25K) 


10.18 


33 


Small Town (pop < 25K but >2500) 


6.25 


179 


Rural (pop < 2500) 


6.94 


217 



Data Source: California Department of Education (see Appendix A) 



Table 4 


California Student Characteristics by EP Teachers 


Emergency permit teachers 
above and below median 


2000 API 
Score 


% African 
American 


% 

Hispanic 


% 

White 


% Low 
SES 


% English 
Language 
Learners 


0-8% EP teachers (below 
median) 


721.07 


5.90 


26.26 


53.61 


37.13 


15.48 
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9-85% EP teachers (above 
median) 


607.09 


10.73 


50.22 


26.37 


58.94 


30.24 


Total 


664.24 


8.31 


38.21 


40.03 


48.00 


22.84 



Data Source: California Department of Education (see Appendix A) 



EP teachers are more likely to find placements in schools that are low-performing, which are more likely to be found in 
urban areas. Table 5 shows a crosstabulation of 2000 API scores divided into deciles and the population status for the 
schools. The table demonstrates that in large cities, schools are far more likely to appear in the lowest deciles of the 
API than in the highest. In the urban fringe (i.e., suburbs) of large cities, more schools are found in the high end of the 
API distribution. 



Table 5 


California API Scores in Deciles by Population** 


API Scores by 
Deciles 


Large City (pop 
> 250K) 


Mid-Size City 
(pop < 250K) 


Urban Fringe of Large 
City (Suburbs) 


Urban Fringe of 
Mid-Size City 


1 st Decile: 346- 
493 


314(146.3)* 


71 (99.9)* 


215(329.2)* 


44 (43.6)* 


2nd Decile: 494- 
542 


202(139.0)* 


107 (94.9)* 


251 (312.9)* 


45 (41.4)* 


3rd Decile: 543- 
589 


158(143)* 


115(97.6)* 


290 (321.8)* 


41 (42.6)* 


4th Decile: 590- 
628 


132(143.0)* 


124 (97.6)* 


299 (321.8)* 


54 (42.6)* 


5‘h Decile: 629- 
665 


106(143.9)* 


123 (98.2)* 


317(323.8)* 


46 (42.9)* 


6th Decile: 666- 
703 


104(141.0)* 


100 (96.3)* 


332(317.3)* 


45 (42.0)* 


Decile: 704- 
741 


99(144.3)* 


94 (98.5)* 


341 (324.8)* 


46 (43.0)* 


8‘h Decile: 742- 
785 


96(141.5)* 


92 (96.6)* 


335 (318.3)* 


55 (42.1)* 


9‘h Decile: 786- 
836 


115(144.6)* 


68 (98.7)* 


395 (325.3)* 


37 (43.1)* 


10th Decile: 
837-969 


101 (140.4)* 


80 (95.8)* 


436(315.8)* 


12(41.8)* 



*First number is observed value; number in parentheses is expected value. 

**Large and small towns and rural areas excluded due to relatively small counts. 

Data Source: California Department of Education (see Appendix A) 

The percentage of EP teachers is correlated with student, teacher, and school characteristics as well, as shown in Table 
6. The Pearson correlation between the percentage of EP teachers in a school and the percentage of students on free or 
reduced price lunch is .439, and the correlation between the percentage of EP teachers in a school and the percentage of 
Hispanic students is .493. Both correlations are significant at the .01 level (one-tailed). These correlations suggest that 
Hispanic students and low-income students are more likely to be taught by EP teachers than students from other ethnic 
groups and students from higher-income families. 



Table 6 



California EP Teachers Correlated with Selected Student, 
Teacher, and School Characteristics 



% EP Teachers 
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% African-American students 


.243** 


% Asian students 


-.094** 


% Hispanic students 


.493** 


% Students on free/reduced price lunch 


.439** 


% Parents not high school graduates 


.378** 


% Parents that attended graduate school 


-.281** 


% 1 St year teachers 


.401** 


School Size 


.148** 



** Correlation is significant at the .01 level (1 -tailed) 

Data Source: California Department of Education (see Appendix A) 

The distribution of EP teachers within districts 

Besides the substantial variation in the distribution of EP teachers across the state, variation is also found within 
districts. A district may have vast differences in percentages of EP teachers found in particular schools. For example, 
the distribution of EP teachers in Visalia Unified School District's elementary schools ranges from zero to 20%, with a 
mean of 6.56% (Note 10). 

In many cases, the percentages of EP teachers are high throughout the entire district. For instance, Buena Vista 
Elementary District, Columbine Elementary District, Ravenswood City Elementary District, and Compton Unified 
School District — all districts serving predominantly minority students — had 1999-2000 averages of 50% or more EP 
teachers for the district. That year, Compton Unified had the dubious honor of having the highest percentage of EP 
teachers in California, with an average of 56.29% EP teachers district-wide, and a high of 85% EP teachers in one 
school! However, as Table 7 shows, there are a far greater number of districts with fewer than 10% EP teachers than 
districts with more than 10% EP teachers. New Haven Unified School District, well known for its emphasis on teacher 
quality and aggressive recruitment of the best teachers (Note 1 1) had the distinction in 1999-2000 of having the 
smallest percentage of EP teachers in the state. However, other districts, such as San Diego Unified, have recently 
changed their hiring policies to eliminate or curtail the numbers of EP teachers hired. San Diego Unified will no longer 
hire teachers on emergency permits. 



Table 7 


Distribution of EP Teachers by District in California 


Number of Districts 


% EP Teachers 


4 


50% or more 


18 


30-40% 


71 


20-30% 


188 


10-20% 


387 


Fewer than 1 0% 



Data Source: California Department of Education (see Appendix A) 



Redesignation of EP teachers as Pre-Interns. 



It is interesting to note that a change in credentialing designations which recently occurred in California has led to the 
appearance of a decrease in EP teachers. The change is the addition of a new category called “pre-intem,” which is the 
next step up from emergency permit on the credentialing ladder. In order to qualify for this designation, teachers must 
have met the EP requirements and they must have enrolled in a teacher preparation program. By encouraging or forcing 
EP teachers to immediately enroll in teacher preparation programs, districts can seem to have fewer EP teachers, while 
in fact they still have about the same number of underqualified teachers. 



Variables Correlating with Student Achievement 



Multiple regression results. 



To examine variables that impact student achievement, a multiple regression was performed using the 1999-2000 API 
data (Note 12). Data for 6,387 California schools was used for the regression. The dependent variable used was the 
school-wide API score. The coefficients are shown in Table 8, The regression demonstrated that factors that are 
significantly negatively correlated with API scores (in order of standardized coefficient beta size) include the 
percentage of students qualified for free/reduced price lunch, the percentage of Hispanic students, the percentage of 
parents without a high school diploma, school size, the percentage of African-American students, the percentage of 
emergency permit teachers, and the percentage of first-year teachers. The factors that were significantly positively 
correlated with API scores were the percentage of parents who had attended graduate school and the percentage of 
Asian students. A production function for this regression yields: 

^Predicted API score^ -.349 Z% free lunch"*^^^ Hispanic students 



-.173 Z% Parents with no HS diploma ••155 "^School size “*055 Z% Emergency permit teachers 



-.031 Z% 1st year teachers +*199 Z% parents completed grad school “•"•076 Z% Asian students 



The R of .905 and the adjusted of .8 19 suggests that most of the variation in test scores at the school level is 
explained by this set of variables (Note 13). With an N of 6,389 and nearly all eligible California schools included in 
the regression (Note 14), these results are significant and interesting (though not particularly surprising) demonstrating 
that most of the variation among schools is accounted for by factors that are beyond the immediate control of schools, 
districts, or the state, including student ethnicity (Note 15), student poverty, and parent education (Note 16), In fact, 
there are only three variables in this model that could conceivably be impacted by state or district actions: the 
percentage of EP teachers and first-year teachers at school sites, and school size. While policy changes could affect all 
of these factors, none are easily changed by simple mandate, . 

The EP teacher coefficient in the regression is significant but small relative to the other coefficients (see Table 8. 
However, it is clear from the regression that the percentage of EP teachers in a school does have an association with 
API scores above and beyond factors such as the socio-economic status of the student body and school size. 
Furthermore, although inexperience and EP status are highly correlated, both exert independent effects on student 
achievement. School size is an additional factor that has been found in other research to influence student achievement, 
along with teacher characteristics. 



Table 8 


Regression Coefficients 




Unstandardized 

Coefficients 


Standardized 

Coefficients 


t 


Sig 


Correlations 


B 


Std Error 


Beta 


Partial 


Part 


(Constant) 


809.527 


2,371 




341,423 


,000 




% African American 
Students 


-1.382 


,067 


-.135 


-20,480 


,000 


-.248 


,109 


% Asian Students 


.828 


.063 


.076 


13,177 


,000 


,163 


,070 


% Hispanic Students 


-.908 


.051 


-.198 


-17.892 


,000 


-.219 


,095 


% Free Lunch 


-1.469 


.048 


-.349 


-30,879 


.000 


-.361 


.164 


% Parents Not H.S. Grads 


-1.181 


.064 


-.173 


-18,423 


,000 


-.225 


.098 


% Parents Attended Grad 
School 


2.006 


.072 


.199 


27.890 


.000 


.330 


,149 


% Emergency Permit 
Teachers 


-.618 


,016 


-.055 


-8,143 


.000 


-.101 


,043 


% 1^^ Year Teachers 


1 

oo 

oo 


.112 


-.031 


-5,242 


,000 


-.065 


,028 


School Size 


•Ml 


.005 


-.155 


-27,463 


,000 


-.325 


,146 



N = 6,387 (schools) 

R=. 905, Adjusted R2 = .819 

Constant (Dependent Variable): Academic Performance Index Score for School 
Data Source: California Department of Education (see Appendix A) 



Policy, Politics, and California Teacher Preparation 



It is rather puzzling that California has such an incoherent, conflicting, and poorly coordinated set of policies for the 
recruitment, preparation, development, and retention of teachers. Cynics who might believe that most state policies are 
incoherent have only to look closely at the efforts of other states to discover that policies do exist for teacher 
recruitment, preparation, and development that are focused and consistent. One example is Connecticut, where there 
has for years been a concerted effort focused on improving teacher quality, rather than a single “silver bullet” strategy 
(Wilson, Darling-Hammond, & Berry, 2001). Connecticut also built on initial policies rather than switching to new 
strategies. This continuity of efforts and the participation of experienced educators in the formation and evaluation of 
policy at all levels helped the state policy maintain coherence over time. Many believe that Connecticut’s improvement 
in student achievement is a direct result of the improvement in teacher quality that resulted from the state's policy. 

It is relatively easy to find examples of California state policies that have served to hurt teacher quality in the state, and 
which have probably impacted student learning as a result. Perhaps the most glaring example is the institution of class 
size reduction (CSR) in the state of California, an event that occurred precipitously and with little thought to collateral 
consequences. Without careful consideration about where teachers would be found to fill the additional classrooms 
created by CSR, the legislation was passed and classes in grades K-3 were limited to 20 students. There is little doubt 
that CSR might be beneficial, at least in early grades, in terms of student learning and teacher and parent satisfaction 
(Mosteller, 1 995). However, the impact of CSR on California's students is less clear, and a recent report has indicated 
that there is no clear causal connection between improved achievement and CSR (CSR Research Consortium, 2002). 
However, RAND researchers have suggested that smaller class size is one of the most important factors in differences 
between math scores on the most recent NAEP assessment (Note 17) (Grissmer, Flanagan, Kawat, & Williamson, 
2000). The Public Policy Institute of California noted the negative effects of CSR on teacher characteristics, stating that 
“CSR led to a dramatic increase in the percentages of inexperienced and uncertified teachers” (p. 1) (Public Policy 
Institute of California, 2002). 

As policy analysts have pointed out, the effects of improvements (in this case, improvements resulting from CSR) must 
be considered in context and in terms of their interaction effects, not in isolation (Hatch, 2000). CSR, a program that 
was intended to be beneficial to students and teachers, opened the door to increasing numbers of emergency permit 
teachers by making jobs readily available. With districts clamoring to hire teachers with or without a credential, CSR 
served as a disincentive for students in teacher preparation program to continue their efforts at obtaining a teaching 
credential through traditional pathways. Teacher training institutions were raided by desperate districts looking to fill 
teaching slots that had been created virtually overnight. In response to the changed teacher training landscape, teacher 
training institutes were forced to immediately expand their course offerings to nights and weekends and change course 
requirements and expectations to accommodate teachers who had already become teachers of record for their own 
classrooms. 



The most unfortunate consequence was that the very children that stood to benefit most from CSR were the ones most 
hurt by the precipitous nature of the policy implementation, for two reasons. The first is that the suddenly increased 
demand for teachers in schools with middle-class students, high test scores, and fewer challenges meant that many 
teachers who had been in more challenging schools were able to take advantage of an opportunity to transfer into these 
“better” teaching placements. A RAND study on California schools found that districts and schools with large 
proportions of Black and Hispanic students had higher initial vacancy rates for teachers (Carroll, Reichardt, & Guarino, 
2000). They also found that teachers tended to transfer away from these schools, and that districts with large 
proportions of Black and Hispanic were not as successful as other districts in recruiting credentialed teachers. Thus, the 
openings that were available for the legions of emergency permit teachers needed to fill classrooms were openings in 
the schools with higher proportions of Black and Hispanic students, which also tend to be schools with higher poverty 
rates and lower student achievement. While the students in these schools gained the benefit of smaller class sizes, 
evidence shows that they became even more likely to be taught by underqualified teachers (CSR Research Consortium, 
1999). 



In terms of preparing emergency permit teachers, there are numerous instances in official California state policy where 
regulations make it difficult to assist EP teachers to be more effective. For example, the state-funded Beginning 
Teacher Support and Assessment (BTSA) program discourages districts from funding the induction of emergency 
permit teachers. BTSA’s website introduction states that BTSA “provides opportunities for fully-prepared first and 
second year teachers to expand and deepen their teaching knowledge and skill" (italics added) (Beginning Teacher 
Support & Assessment, 2000). However, the idea that “fully-prepared” teachers should be given priority in terms of 
funding and slots in the program seems like a contradiction in terms. The first of BTSA's stated purposes is to "Provide 
an effective transition into the teaching career for first- and second-year teachers in California" (under "BTSA - 
Basics"). The second purpose is to "Improve the educational performance of students through improved training, 
information, and assistance for new teachers." If these are the primary purposes of BTSA, it seems that emergency 
permit teachers need the support of BTSA at least as much as teachers who are already much farther along in their 
training. Further, once EP teachers have completed two or three years of teaching while attending school at night and 



on weekends, they will be understandably reluctant to participate in BTSA, a program designed for “first- and second- 
year teachers.” However, it is also clear that reshaping BTSA to include EP teachers would result in a program that 
would be less useful to traditionally-prepared first- and second-year teachers. 

Another example of ways in which policy conflicts are making it difficult for EP teachers to receive the training and 
support they need is that pre-intems and EP teachers can legitimately teach in a classroom as the teacher of record, but 
they are not permitted to student teach until they have passed their subject matter requirements. Thus, an EP teacher or 
pre-intem is not allowed to student teach under the supervision and guidance of an experienced teacher, but they are 
allowed to teach a class by themselves, without support! 

Additionally, California has provided scholarships for teachers to support them as they complete their teacher 
preparation program and obtain their credentials. But the funding available for these scholarships is adequate only to 
support a few hundred teachers each year. Consider that there were about 10,000 new EP teachers hired in California 
last year, many (perhaps most) of whom chose an alternative credentialing route because they did not have sufficient 
funds for traditional teacher preparation. The funding provided for helping teachers complete their preparation 
programs is clearly inadequate, even though reducing the numbers of EP teachers is a priority for the state, judging by 
the legislation addressing the issue, as will be discussed in more detail in the next section. 

Legislative Solutions to the Emergency Permit Problem 

As noted previously, most of the variables negatively correlated with API scores are beyond the direct control of 
policymakers. The legislature cannot easily solve child poverty, which explains most of the variance in test scores at 
the school level. They can, however, try to impact teacher quality and thus indirectly help student achievement. Some 
legislation has sought to mandate reductions in emergency permits without responding systemically to the underlying 
problems that have produced the shortage of qualified teachers and the maldistribution of underqualified teachers (Note 
18). 



In the 2001-02 Regular Session of the California Legislature, several bills were introduced in an attempt to impact the 
distribution and/or preparation of EP teachers, to increase the overall number of qualified teachers, or to increase the 
percentages of credentialed teachers in low performing schools. Table 9 summarizes these bills and their status (Note 
19). Some of the bills that seek to eliminate or reduce the percentages of EP teachers in schools are problematic. 
Without state support and incentives, the goal of eliminating EP teachers would be difficult or impossible for many 
districts, particularly in the four California districts where 50% or more of all teachers are teaching on emergency 
permits (Note 20). 



Table 9 


2001 California Legislative Proposals Related to EP Teachers 


Bill and 
Author 


Status 


Summary 


AB 833 
(Steinberg) 


Vetoed by 

Davis 

10/5/01 


Establishes the Public School Teacher Qualification Equity Program which 
provides for a teacher qualification index (TQl)*. The bill would require that 
school districts calculate a TQl for each school and make efforts to increase the 
TQl for each school until it obtains a specified rating. The original version of the 
bill called for specific interventions if the districts did not meet its TQl goals 
within a certain length of time, but this part was taken out of the final submission 
of the bill. 






*TQ1 is based on the number of underqualified teachers within the district and 
within each school. Fewer underqualified teachers result in higher TQl scores. 


AB 721 
(Steinberg) 


Under 

submission 


Establishes the Teachers for Low-Performing Schools Renewable Grant Program 
designed to encourage postsecondary institutions with teacher preparation 
programs to “recruit, prepare, and support new teachers to work and be successful 
in low-performing schools.” The bill is premised on the belief that low-performing 
schools suffer particularly from a shortage of credentialed teachers. 


SB 57 (Scott) 


Approved by 
Davis 9/8/01 


Clears the way for quicker credentialing of private school teachers who decide to 
teach in public schools and for district interns to complete their credentialing 
program early if they pass certain assessments. 


SB 743 
(Murray) 


Vetoed by 

Davis 

10/13/01 


Provides funds and mandate for the California Commission on Teacher 
Credentialing to address the issue of emergency permit teacher distribution and 
make specific recommendations to districts with large numbers of EP teachers in 
low-performing schools in low-income communities, including recommendations 
for recruitment and retention policies. 
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SB 837 (Scott) 


Approved by 

Davis 

10/5/01 


Requires school districts to meet specific requirements in a diligent search for 
certificated teachers. Documentation will be required from districts hiring teachers 
on emergency permits to demonstrate that they made a diligent search for 
credentialed teachers prior to offering positions to non-credentialed teachers. 


SB 508 

(Vasconcellos) 


Suspended by 

Assembly 

9/12/01 


Extensive set of recommendations to assist “California Unrealized Learners,” i.e., 
schools in the lowest two deciles of the API. Provides for additional funding as 
well as improved recruitment and retention strategies in the form of extra salary 
and bonuses, professional development, extended school year, outreach and 
assessment consultants, benefits to highly qualified teachers who teach in these 
schools, loan assumptions for new credentialed teachers, limits on the number of 
underqualified teachers at the schools, smaller class size, etc. 


SB 321 
(Alarcon) 


Unfinished 

business 

10/15/01 


Authorizes Los Angeles Unified School District to create a pilot program for 
offering a 30-day training session to all emergency permit teachers who are 
assigned to schools have 20% or more teachers on emergency permits. The 
training would take place before the teachers began teaching. 


SB 319 
(Alarcon) 


Set for 
hearing 
1/16/02 


Amends the Teaching as a Priority Block Grant that awards grants to schools 
districts to attract credentialed teachers for low-performing schools. The 
amendment would require that the school district meet its API performance goal 
and that the district agrees to increase credentialed teachers in all schools to 90% 
or more in order to be eligible for grants. 



AB833 

The authors of A B 833 proposed a Teacher Qualification Index (TQI) that is comprised of two separate school scores. 
A school would receive a “quantity rating” of “10” at the high end of the spectrum if it had fewer than 5% 
underqualified teachers, and a “1” at the low end of the spectrum if it had 45% or more underqualified teachers. A 
school would receive a “distribution rating” of “10” at the high end if its percentage of underqualified teachers was less 
than or equal to the average percentage of such teachers for the entire district, and gamer a “1” if its percentage of 
underqualified teachers was greater than 80 percent more than the average for the district. A special adjustment would 
be made for schools with low percentages of underqualified teachers, since it would be possible for a school with only 
4% underqualified teachers to get the lowest distribution rating in a district where the average was 1%. An average of 
the two rafings provides the single number designated TQI. In order to raise a particular school's TQI, the number of 
underqualified teachers would have to be reduced at the school to improve the quantity rating, and would have to be 
reduced in comparison to the district average in order to improve the district average. Thus, the district is provided with 
an incentive to redistribute both credentialed and EP teachers within the district more evenly. 

This bill encourages districts to equalize placements of EP teachers among schools, a legislative act that might carry 
some weight at the bargaining table. Teacher unions might agree to stricter transfer rules in order to prevent the 
continual turnover in low-performing schools that would result in the schools being constantly out of compliance. 



However, as the bill evolved, it lost its enforcement section. Districts that were out of compliance would have had to 
submit to an evaluation of the problem and development of a Teacher Quality Improvement Plan (TQIP). Subsequent 
to this examination, the Fiscal Crisis and Management Assistance Team might also conduct their own investigation and 
recommend changes in the district in an attempt to achieve compliance. According to earlier drafts of the bill, some of 
the actions the State Board of Education might then mandate were 



• class size reduction in certain schools and grades; 

• pay incentives for teachers willing to work at certain schools; 

• housing subsidies for teachers willing to teach at selected schools; 

• increased funding for materials, books and technology at certain schools; 

• facility improvements; and 

• focused staff development for beginning teachers. 



While these measures alone might not have resolved the teacher distribution problem, they might have alleviated some 
of the reasons for high teacher turnover, thus providing hope for maintaining a stable staff as the new teachers gain 
experience and the EP teachers achieve their clear credentials. However, after the language regarding the TQI was 
deleted, the bill passed but was vetoed by Governor Davis. 

SB 508 

This is a comprehensive proposal that attempts to work on some of the systemic problems underlying poor student 
performance in the state (as defined by the two lowest deciles of the API). The belief that is demonstrated throughout 
this bill is that good teaching matters. The bill focuses simultaneously on a number of important aspects of improving 
the quality and distribution of the teacher workforce in districts with low-performing schools, including: 
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• improving recruitment and retention, 

• providing increased salaries and bonuses for highly qualified teachers, 

• offering professional development for teachers in low-performing schools, 

• extending school years in low-performing schools with extra days being used for professional development 
and for developing working relationships with parents, 

• offering extra benefits for highly qualified teachers willing to work in low-performing schools, 

• providing outreach consultants to help schools develop working relationships with parents and community 
members, 

• providing assessment consultants so that schools can better understand where their students are failing and thus 
develop measures to directly address the areas of greatest need, 

• providing loan assumption opportunities for new credentialed teachers (not EP teachers) willing to teach in 
low-performing schools, 

• setting limits on the numbers of underqualified teachers at low-performing schools, 

• reducing class size in low-performing schools. 

These measures are intended to act in concert to assist low-performing schools in improving their achievement (as 
measured by the API). The combined effects of these measures might in fact go a long way towards helping these 
schools, but they will be costly. Besides the bonuses, incentives, and loan reductions payable directly to teachers, there 
would be substantial costs for reducing class size, hiring consultants, and extending the school year. The current 
economic crisis in California (and in the nation) will make such expensive measures less likely to meet with approval. 
Whatever the fate of the bill might have been before September 11^^, the fact that the bill was suspended by the 
Assembly on September 12^^ suggests that it was seen as too sweeping and too expensive to gain acceptance in the 
current economic climate. The bill remains in suspension, lacking appropriations. 

Two pieces of legislation that were approved by Governor Davis require little in the way of additional expenditures, 
but their impact on the hiring and distribution of EP teachers will probably be slight. SB 57 makes it easier and faster 
for private school teachers to get their credentials to become public school teachers in California. The bill also allows 
district interns to take certain assessments (performance and written) that will allow them to gain their credentials 
earlier, though exactly what these assessments are and who will judge acceptable completion of the required 
assessments is not clarified. SB 837 mandates that districts meet specific requirements in a diligent search for 
certificated teachers. Documentation of the districts' efforts in this search will be required from districts who hire EP 
teachers in order to demonstrate that they made a concerted effort but were unable to find and hire qualified teachers. 

An Alternative Proposal for Reducing the Number of EP Teachers in California 

Another approach would be to focus efforts on improved retention rather than focusing most of the efforts on the 
teacher labor market, through the following mechanisms. 

1. Help EP teachers quickly obtain their clear credentials 

Many EP teachers in California work for years to get their clear credential. While districts may be willing to help these 
teachers, most districts are not providing the sort of guidance and assistance that EP teachers need in order to surmount 
the obstacles. EP teachers interviewed for a report sponsored by the Bay Area Consortium for Urban Education 
(BACUE) expressed a longing for more assistance through the district, the county office and the universities where 
many of them were taking the courses they need for their credential (Goe, Castro, & Curry, 2001). One way to help 
these teachers would be for the state to provide funds to districts with high percentages of EP teachers to develop pilot 
programs to assist these teachers in more quickly getting their credentials. Districts that create very successful 
programs (as judged by how quickly and effectively they were able to prepare high-quality teachers and move them 
from EP to clear credential status (Note 21)) could then share their models with similar districts. 

Many EP teachers are valuable to schools for a variety of reasons. In a state with a very diverse student population, 
hiring and keeping minority teachers is a priority, and EP teachers are more likely to be minorities than traditionally 
certified teachers. In addition, many EP teachers are willing to teach in urban schools and schools with high 
percentages of minority students, and retaining teachers in those schools is crucial. EP teachers in California are more 
likely to be male than the currently certified teachers, and are teaching in subjects where there is a great need (such as 
math, physical science, and special education) in higher percentages than are regularly certified teachers (Note 22). 
Thus, the state could benefit from finding ways to assist and train these teachers while moving them efficiently through 
the system to obtain their clear credentials. 

One possibility is to create a second branch of California's Beginning Teacher Support and Assessment system (BTSA) 
specifically for EP teachers. Many EP teachers already participate in BTSA, though they are discouraged from doing so 
because BTSA was designed for teachers who had completed a traditional teacher preparation program. The advantage 
to creating a second branch of BTSA for EP teachers would be to create improved regulation of teacher training for EP 
teachers, while extending to them the support and mentorship new teachers receive through BTSA. Under the existing 
system, EP teachers have no provisions for training, support, and assessment other than what is provided by school 
districts (Note 23) and by colleges where teachers are taking courses in order to earn their clear credential. This results 
in substantial variability in the types of support and training that these EP teachers are receiving. Providing a regulated. 




uniform set of standards for the training of EP teachers could be of great benefit to the EP teachers as they work 
towards their clear credentials. 

II. Create incentives for experienced, credentialed teachers to teach in low-performing schools 

Some states, including Connecticut, have legislated that supplementary grants be given to poorer school districts to 
enable them to hire and retain high quality teachers (Wilson et al., 2001 ). Salary increases or one-time bonuses could 
be offered as incentives to attract experienced, qualified teachers into low-performing schools with high turnover and 
high percentages of EP teachers, though the effectiveness of such incentives remains to be seen. In California, the 
Teaching as a Priority (TAP) grant program serves a similar purpose, providing allocations to schools with an API rank 
of 1-3 with $44.00 per student and schools with an API of 4-5 with $29.00 per student. 

However, districts must apply for the grants on a competitive basis, and the funding is limited. In addition, there are 
few regulations on how the district may use the funds, other than the requirement that the funds must be used for 
“retention and recruitment of credentialed teachers to work in low-performing schools” (California Department of 
Education, 2001). Thus, districts, working either independently or in consortiums, may use the funds for signing 
bonuses, housing subsides, vested annuities, and improving working conditions. Obviously, the ways in which districts 
will use funds will vary considerably, and it remains to be seen how successful these funds will be in attracting 
credentialed teachers to underperforming schools. It is possible that different uses for the funds are differentially 
successful in attracting teachers. Geographic differences may also contribute to the success of various uses of the 
funds, with housing subsidies undoubtedly appealing to teachers in areas with soaring housing costs, such as San 
Francisco. 



It might be most productive to create financial incentive packages for experienced, credentialed teachers that include 1) 
a contract requiring a certain number of years of service in the low-performing school, and 2) an agreement to mentor 
an EP or novice teacher. This requirement would help stabilize teacher turnover in low-performing schools by building 
a base of teachers who are committed to staying with the school and by providing assistance to EP or novice teachers 
who might then be more successful in their first years of teaching and more willing to stay at the school. An added 
bonus of requiring mentoring is that mentoring a new teacher within the BTSA program as a support provider has been 
found to be very beneficial to the mentoring teacher (Wing et al., 2002). Thus, both the beginning teacher and the more 
experienced teacher are likely to benefit from the relationship. 



Another approach which could be tried is paying teachers at underperforming schools a salary based on a longer work 
day. The justification is that the teachers at these schools have a more challenging workload and thus spend more hours 
on the job. The difficulty is that all teachers at such schools are assumed to work longer hours, including the EP 
teachers. Thus, it may be difficult to justify differential pay for only the credentialed teachers. 

III. Emphasize new teacher retention 

The state might provide low-performing schools with substantial targeted funds that would be used solely to provide 
bonuses to teachers designated by the schools as “keepers,” i.e., teachers that the principal (perhaps with the help of an 
advisory committee of teachers and parents) felt had the potential to become excellent teachers at that school. This 
would allow the principal to go beyond simply asking particular teachers to stay for the good of the school by offering 
the teachers an incentive to stay for their own good as well. The bonus would thus provide the “keepers” with clear 
affirmation that they were considered valuable to the school, and might provide enough of an incentive to prevent their 
transferring to other less challenging schools. In addition, new teachers are lower on the salary scale and thus might see 
even relatively small bonuses as substantial incentives, meaning that this could be a cost-effective strategy. Those new 
teachers who did not receive the bonuses would feel free to move on to other schools. It should be clarified that the 
“non-keepers” would not necessarily be “bad” teachers. Rather, such teachers might be poor matches for the particular 
schools they were in, and a transfer would give them an opportunity to find schools that were better suited to their 
particular talents and needs. 

Most bargaining agreements allow transfers after only one year of teaching. Keeping the best new teachers in their 
initial placements could greatly benefit low-performing schools by providing a stable faculty of teachers beyond the 
first year. Such a faculty may increase in experience, knowledge, and cohesiveness each year, all of which may benefit 
students. As an added bonus for these schools, school reform efforts that are hampered by continual turnover of staff 
will have an improved chance of success. In addition, each year after the first year, teachers would earn an additional 
bonus if they chose to remain, up through the first five years. 



rv. Provide adequate resources, training, and facilities for low-performing schools 



The items listed in AB 833 as the “sanctions” for districts that are unable to achieve compliance under the TQI plan 
should be provided to underperforming schools as a matter of course, not as a reward or a punishment. It is clear from 
the regression shown in this article that most of the variance in student test scores at the school level appears to be 
associated with variables that are beyond the control of the state, district, or school. Districts with high percentages of 
minority and low-income students may have little flexibility in teacher hiring, and may not be able to avoid hiring EP 
teachers. Rewarding or punishing districts based on their test scores or their hiring practices flies in the face of these 
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realities. If the state provides better support, more resources, improved training, and adequate facilities at low- 
performing schools, teacher attrition should decrease, and thereby lower the demand for additional teachers. As 
demand falls, it will be possible to become more selective in hiring practices so that fewer EP teachers will be needed. 
With less demand for EP teachers, more teachers will seek credentials through traditional teacher preparation 
programs. The requirements for entering teaching under an emergency permit could also be augmented at that point to 
discourage all but the most serious applicants from taking an emergency route to their credentials. 

In addition, schools with high percentages of minority students and/or poor students should receive additional 
resources from the states for targeted professional development aimed at better equipping teachers to understand and 
work with the challenges that their students may face. Many teacher preparation programs inadequately prepare 
teachers for these challenges, resulting in teachers leaving the profession because they are unable to cope with the 
demands. Teaching in high-poverty schools can be rewarding, and with appropriate training and support, teachers can 
discover the rewards and overcome many of the frustrations. One approach to preparing teachers for working in low- 
performing schools is to assign some teachers (Note 24) to these schools for student teaching, under the guidance of a 
strong, effective teacher. While inexperienced teachers may be reluctant to take on the challenges of a low-performing 
school, a student teaching experience with a strong supervising teacher might give them an opportunity to see for 
themselves that good teaching and learning is possible even in low-performing schools, and to develop strategies that 
would increase their confidence that they, too, could be successful in such a setting. 

V. Provide more state funding for teacher preparation scholarships 

Since many EP teachers choose to complete their teacher credentials while they are teaching because of financial 
burdens, offering greater incentives for them to complete traditional programs may discourage this trend. However, the 
incentives should be focused on recruiting more minority teachers to traditional teacher preparation programs, since 
California's teaching force is very short of minority teachers, particularly given the high percentage of minority 
students in California's schools. And since minority teachers are more likely to take an alternative route into teaching 
than to take the traditional route, it is important to consider providing additional incentives to recruit high-quality 
minority candidates into traditional teacher preparation programs. 



VI. Place EP teachers in high-performing schools with reduced loads 



There may be useful reasons to keep teachers currently on emergency permits as part of the teaching force for many 
reasons. However, their current pathway into full credential status usually starts in an underperforming school, an 
unfortunate starting place for both the EP teachers and the students, whose poor test scores may indicate greater 
academic needs. In these underperforming schools, they are usually relegated to the grades and classes that the teachers 
with more seniority do not want. They are typically assigned full class schedules, with no release time for observing 
other teachers, meeting with mentor teachers (if they even have one), etc. They are usually placed in schools that are 
particularly challenging in terms of high percentages of low-SES students or English language learners. And they are 
usually placed in schools where greater-th an -average percentages of the teachers are also teaching on emergency 
permits, are new teachers, or both. Thus, the schools' capacity to help these beginning teachers is limited by their 
inability to provide on-site mentoring and support. 



Most bargaining agreements do nothing to prevent this placement of EP teachers at the most challenging low- 
performing schools, but in at least one case (Los Angeles) there have been efforts to ensure that at least the EP teachers 
are not all concentrated in high numbers at particular schools, and that teaching experience is more spread out among 
schools. Such efforts were likely the result of long negotiations, but they are at least a possibility for other districts. 
Legislation that mandated a lighter load for EP teachers (60-70% of the regular teaching load) accompanied by 
intensive training programs (under the umbrella of district induction programs) designed specifically for EP teachers 
could go far to ensure that these teachers are quickly brought up to speed and that they develop the talents and 
knowledge that put them on the road to becoming excellent, credentialed teachers. 



Putting EP teachers on reduced loads throughout the state is certainly an expensive proposition. One factor in making 
the spending of additional resources more cost effective would be to adopt a graduated schedule for EP teachers, so 
that their greatest reduction in teaching load would occur their first year in the classroom, and teaching loads would 
increase each year after that. Progress towards their credential would be closely monitored and remediation supplied as 
needed. 



However, given the expense of such a plan, it is essential that regulatory mechanisms are built into the system that so 
that the reduced load and additional support provided by EP teachers does not become an incentive for teachers to 
bypass traditional teacher training programs. Emergency permits were intended to serve districts in emergency 
situations, and this method of beginning teaching was never intended to be institutionalized as a legitimate alternative 
to traditional teacher preparation. If California were to subsidize the preparation of large numbers of EP teachers with 
reduced loads and additional help, safeguards should be developed to avoid institutionalizing this as a credentialing 
mechanism. One safeguard would be that districts would have to demonstrate that they made every possible effort to 
recruit and hire credentialed teachers. 

Further, it is essential that specific demands on teachers to progress towards their credential be set and adhered to; 
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otherwise, it might be tempting for teachers to enter the profession on an emergency permit to take advantage of the 
comparatively lighter load. By demanding demonstrated progress on a set schedule, it would be clear to teachers that 
the advantages of a lighter load were far outweighed by the progress requirements. 

In addition, in the interest of recruiting more minority teachers and bilingual teachers, the plan could be offered only to 
teachers meeting certain district-level needs. For example, in a district with a very high percentage of Spanish-speaking 
students, priority could be given to bilingual Spanish/English emergency permit teachers. 



Further, in the interest of cost-effectiveness in a cash-strapped state, it is important that state subsidies for assisting EP 
teachers be used for teachers that are genuinely interested in a career in teaching, as opposed to those who are just 
“trying out” the profession to see how it suits them. Teachers who wanted to take advantage of EP status and support 
should demonstrate, perhaps through a contractual agreement with the state or a particular district, that they are serious 
about teaching as a profession, committing to a minimum of three years of classroom teaching in exchange for state 
and district support. This would exclude short-term teachers such as those placed on two-year assignments through 
Teach for America in hard-to- staff schools (Note 25). 



VII. Focus class-size reduction funds on underperforming schools 

When California policymakers decided to approve over a billion dollars a year for class-size reduction, they restricted 
the funds to grades K-3, but made no requirements in terms of school performance for the funds. Thus, high- 
performing schools with upper- and middle-class students were entitled to the same small classes as schools with high- 
poverty students or mostly English language learners, students who ostensibly need extra teacher attention much more. 
Most studies of class-size reduction find greater benefits for minority students and for high-poverty students than for 
students with high socioeconomic status. While CSR is a popular reform, its cost-effectiveness in terms of student 
benefits for middle- and upper-income students has not been demonstrated. However, teachers are decidedly in favor of 
smaller classes. 

By limiting CSR to K-3, California has effectively concentrated many of its best multiple-subject teachers in these 
grades, since teachers with more seniority often have a greater choice in what grade they will be teaching. Teachers in 
grades four and five do not receive the CSR benefits, so there is a sharp division between what both teachers and 
students experience at the classroom level in the early grades as contrasted with grades four and five. My proposal 
would encourage teachers who desire smaller classes to remain in (or transfer into) underperforming schools by 
prioritizing CSR funding for schools in the bottom deciles of the state's Academic Performance Index. 

The way CSR is currently structured, even schools that already had fewer than 20 students qualified for the CSR funds, 
so districts that already had small class sizes (suggesting that they had an ample supply of teachers as well), qualified 
for and received the additional funding for making no change whatsoever in class size! From a cost-effectiveness 
standpoint, such a policy is seriously flawed. Further, districts have always been able to choose to implement CSR in 
only certain schools, and even in certain classrooms, as long as they follow the state's requirement of implementing 
CSR first in grade one, then in grade two, and then in either kindergarten or grade three (Note 26). But since the state 
provides no incentives whatsoever to implement CSR only in certain grades or in certain classrooms; districts tend to 
implement it across the board if they can find enough teachers (credential ed or otherwise) to staff the classrooms, 

I propose that California adopt new legislation that would phase out CSR in high-performing schools with stable 
teaching forces and phase in CSR at grades four and five in low-performing schools. While there are no guarantees that 
this would improve student achievement at these schools, it might reduce teacher turnover at underperforming schools 
by reducing teacher workloads and increasing teacher satisfaction. Likewise, it would be an incentive for teachers to 
choose to teach in (or transfer into) underperforming schools if they wanted to teach smaller classes, or to remain in 
these schools rather than transferring to higher-performing schools, thus stabilizing the teaching force and reducing the 
need to hire new or emergency permit teachers. Optimally, a balance would be achieved during the phase-in that would 
reduce the statewide need for additional teachers to support CSR, which triggered the influx of EP teachers in the past 
few years. New teachers will continue to be needed in large numbers in California due to the growing number of 
teachers nearing retirement age and the burgeoning school-age population (at least in some areas of California), It is 
hoped that a combination of traditional teacher preparation programs and selective, subsidized, well-supported 
emergency permit programs could supply sufficient numbers of new teachers to meet the demands. 

Coherency and Coordination of Policies 

The policy solutions suggested in the previous section focus on using various forms of incentives (support, mentoring, 
release time, monetary awards, targeted class-size reduction, and training opportunities) to: 



• increase retention in the profession and in underperforming schools; 

• increase teacher commitment to underperforming schools; 

• increase enrollment in traditional teacher preparation programs, particularly among minority and bilingual 
teachers; 

• provide additional supports to EP teachers and move them systematically towards full credential status. 
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This set of solutions attempts to build upon structures that are already working successfully in California, phase out 
programs or policies that are ineffective, and focus resources where they are most likely to benefit students. As a 
coordinated effort, such policies would have to be implemented carefully and systematically, with constant reference to 
both collateral effects of the policy and to the context for implementation (i.e., ensuring capacity before stressors are 
added to existing systems). 

Conclusion 

California's accountability system has created an interesting opportunity to better understand what matters most in 
school achievement. Researchers and policy makers, as well as the general public, can now clearly connect student 
achievement (at the school level) with a number of other variables, including the percentages of underqualified 
teachers. Seeing these connections, like the stair-step pattern of test scores and teacher credentials in Figure 1 presented 
above, can be shocking. However, it is important that policy makers and legislators do not rush to judgment and 
condemn all EP teachers as the cause of poor student achievement. Rather, both the EP teachers and the students are 
victims of poverty and state policy. Poverty cannot be so easily addressed, but state policy can be. State and district 
policies have made it necessary, possible, and even desirable for teachers with limited funds to begin teaching before 
they are fully credential ed and then to transfer to “better” schools when they get seniority. State policy has also allowed 
conditions in some California schools to deteriorate to such an extent that both teachers and students are depressed and 
frustrated by their teaching and learning conditions, as evidenced by the recent class action suit against the state of 
California filed by students in underperforming, underresourced schools (Sahagun & Helfand, 2000). 



There can be no question that all students should have a well-qualified and highly trained teacher, and California 
should work towards that goal at a reasonable pace, using incentives to encourage redistribution of experienced, 
credentialed teachers into low performing schools and to encourage more teachers to complete their full credential 
before taking full responsibility for the education of California's children. In particular, minority teachers should be 
aggressively recruited into teaching through traditional teacher preparation programs where they can be better 
supported and provided with added incentives to encourage them to make teaching a career. Finally, researchers into 
the connection between teacher quality and student achievement should focus on clarifying the skills, qualities, and 
characteristics that a full credential represents, and should ensure that teacher preparation programs, credential ing 
mechanisms, and induction programs focus on building those skills, qualities, and characteristics. 

Notes 

1 . Shields et al, shows 14% of CA teachers on emergency permits or waivers (Shields et al,, 2001 ). The difference in 
these figures is likely due to data sets used and how teacher credentials are designated. 



2. It is widely believed that some California districts have “dysfunctional” hiring processes which result in missed 
opportunities to hire qualified teachers. Interest is growing among education reform organizations and those who 
sponsor reform efforts in California to consider “deep” human resources reform as a mechanism for improving district- 
level teacher recruitment and retention. 



3. Academic Performance Index scores are single scores given to California schools based on the SAT-9 scores for 
students in grades 2-11. The Index will eventually include other measures, such as scores on the state standards tests, 
but these measures are still being tested for reliability and validity. The Index is composed of weighted SAT-9 scores 
so that students moving from lower quintiles of the SAT-9 into higher quintiles earn more points for the school than do 
schools moving from middle quintiles into higher quintiles. For further information on the API, see the California 
Department of Education website: http://api.cde.ca.gov/. 



4. Since true experimental designs are rarely possible in education, evidence is generally developed through 
triangulation of studies using a range of correlation methods and quasi-experimental designs. 



5. “Underperforming” for API purposes means that the schools received a score of 5 or below in the API, i.e., that they 
are performing in the bottom half of the API statewide. 

6. 35% of California teachers had a master's degree (data year 1999-2000). 

7. Experienced teachers can serve as mentors and support providers to less experienced teachers. They are also more 
likely to feel competent to serve on school site committees and assist in leading professional development sessions. 
And they are important for school continuity in reform efforts. 

8. However, the results of this study controlled for teacher preparation status. It could thus be argued that the transfer 
patterns would be different for teachers depending on their credential and preparation status. 
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9. Using 1999-2000 school year data. The 2000-01 data shows a slight decrease in the number of EP teachers to about 
10%, though there has been an equivalent increase in the numbers of pre-intems. 
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10. 1999-2000 data. 



1 1 . For a description of New Haven's strategies, see Snyder, J. (2000). New Haven, California's Teaching Quality 
System: What States Can Learn from One District's Success. The State Education Standard, 7(1), 7-1 1 . 

12. Additional details about the regression can be found in Appendix B. 

1 3. It is important to note that because the data are aggregated at the school level rather than the individual level, there 
is undoubtedly more unexplained variation within schools than is captured by this between school analysis. 

14. Schools that were missing API scores or missing data for one of the other variables were omitted from the 
regression. 



15. Since student ethnicity is highly correlated with socio-economic status in California, it is likely that ethnicity serves 
as a proxy for SES in the regression. 

16. Parent education probably serves as a proxy for socio-economic status in the regression. 

17. The other factors named by the authors are higher rates of pre-kindergarten attendance and “more of the resources 
necessary to teach.’’ 

1 8. Many underlying problems have been listed in the literature on teacher recruitment and retention, including: lack of 
incentives to attract new teachers, salaries that are not competitive with other jobs teachers might qualify for, the 
requirement of additional education beyond a baccalaureate degree, school conditions that teachers find frustrating, de- 
professionalization of teaching, and recently, high-stakes testing and the resulting emphasis on test preparation. 



19. Status as of December, 2001. Full text versions of the bills and their histories are publicly available at 
http://www.leginfo.ca.gov/bilinfo.html. 



20. In 2000-01, Compton Unified had 56% EP teachers, and three elementary districts had 50% or more EP teachers: 
Ravenswood City, 55.1 1; Columbine and Buena Vista, 50% each. 



21 . It is important that the effectiveness of a program designed to move teachers quickly towards a clear credential is 
judged not only by its speed but by the quality of the resulting teachers. Effectively prepared teachers should be as 
competent as teachers prepared in traditional teacher training programs. 



22. However, it must be noted that EP teachers may not be credentialed in the field in which they are currently 
teaching. In other words, they may be teaching math because they were the only teacher available, not because they 
have a degree and training in teaching math. 



23. Some districts have redirected PAR (Peer Assistance and Review) funding to provide support to some new 
teachers, thus ensuring that every teacher receives the support they need, regardless of their credential status. PAR was 
originally formed to provide assistance and mentoring to struggling teachers who had been in the teaching force for a 
while, under the assumption that new teachers would receive ample support from BTSA. But the burgeoning 
population of EP teachers caused many districts to prioritize the funding for new teachers. 



24. It would be important to ascertain initially whether the student teacher had an open mind about teaching in a low- 
performing school, so as not to waste the opportunity of working with a strong teacher on a student teacher with a 
confirmed disinterest in such schools. 

25. This is not meant to penalize TFA teachers, but only to ensure that resources are spent in a cost effective manner, 
i.e., where there is a likelihood that they will contribute most to the development of a pool of qualified, permanent 
teachers for California. It is also assumed that TFA teachers have other means of support and mentoring through the 
TFA organization. 



26. For further information on CSR and the rules governing its implementing, see the CSR page on the CDE website: 
www.cde.ca.gOv/classsize/sy0102/question.htm 
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Appendix A 



Data Used in this Study 

Data sources. Title I reporting requirements as well as the recent push for school, district, and state accountability for 
student achievement has resulted in many states offering public access to the data used in making judgments about 
schools. California is no exception. Starting in 2000, the California Department of Education (CDE) began posting data 
files containing school-level variables and Academic Performance Index scores on its website. These files are available 
for downloading by the public. Other databases with school-level information are also available. 



For this study, variables were analyzed from a number of different data sets. All data files are available at the CDE 
website, www.cde.ca.gov/demographics (teacher and school variables data files) or at 

http://api.cde.ca.gov/datafiles.html (API data files). Some of the files are updated by the CDE regularly during the year 
or as new information becomes available. Thus, the files downloaded at one point in time may have undergone 
changes. The sources for the files used for this study are as follows: 



• Teacher credentials and experience (tchcrd99). Contains a breakdown by school of teacher experience and 
credentials, aggregated at the school level into total numbers and/or percentages. 

• Profile of certificated staff, by school (prcert99). Contains school-level aggregated variables on gender, race, 
degrees held, age, and years of service in education. 

• List of California public school districts and schools (pubschls). Contains information on schools such as grade 
span, charter status, and categories of populous areas such as urban and rural. 

• Teachers (full-time equivalent) by subject area and school (teasch99). Contains numbers of teachers in 
secondary subject areas, special education, etc. 

• API (api2Kbdbf). Contains API scores for 1999 and 2000, along with breakdowns of scores by student race 
and other designations such as socio-economicaily disadvantaged and English language learners. Also contains 
parent education variables and percentages of teachers with full or emergency credentials. More recent API 
scores for 2000-01 school year are also used, but less extensively because this article was nearly finished when 
they were made available. The file is apiOlg.dbf 

• PAIF (paif97_98, paif98_99, paif99_00, paif00_01). Staff characteristics by record identification and CDS 
code from the CBEDS Professional Assignment Information Form (PAIF). Contains information on all 
California teachers, identified with a unique record identification code that changes each year. Provides 
teacher-level information on gender, credential status, ethnicity, education level, years of service, and subject 
authorized to teach. 
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Data Analysis. All files were downloaded as zipped dbf files to a personal computer. After unzipping, files were 
opened in SPSS statistical analysis software. Besides downloading files directly from the internet, a number of 
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variables were also created using other sources of information or using the compute function in SPSS. Figure 1 was 
created from statistics generated in SPSS and copied into Excel XP software. 

Appendix B 



Multiple Regression 



Number of schools. The entire population of 6,389 elementary, middle, and high schools with valid API scores 
(dependent variable) and valid data for independent variables for the 1999-2000 school year were included in the 
regression. Fewer than 1 ,000 schools were missing relevant data and were excluded from the regression. 



Dependent variable. The dependent variable is the schooMevel API Score (Academic Performance Index), which is the 
score received by participating schools for the SAT*9 test taken in spring 2000. For further information about the API, 
see the California Department of Education (CDE) website: 

http:/Avww. cde.ca.gov/psaa/api/yeartwo/base/api infogb.pdf. The SAT-9 test is given to students in grades 2-11. The 
CDE then weights the scores for various subjects (greatest weight is given to reading and math), and weights the scores 
according to which of five performance bands students are in. Subgroup scores for significant subgroups (based on 
ethnicity, English language ability, and socioeconomic status) are calculated separately and are available in the data set. 
For further information about the calculation of API scores, see the CDE website 
http://www.cde.ca.gov/psaa/api/yeartwo/base/apicalc.xls. 

Independent variables. There are a limited number of variables to choose from in the publicly accessible API data base, 
including student race, parent education levels, free lunch eligibility, English proficiency, student enrollment in grades 
tested, and teacher credentialing status. All variables were used directly as reported in the data file downloaded from 
the CDE except for the school size variable. The school size variable was computed from the enrollment in grades 2-1 1 
reported by the CDE divided by the number of grades in the school. The decision to compute this number and use it as 
the school size was made because California schools vary widely in grade configuration, with some schools having 
only one or two grades and others having six or more. The computed school size thus gives a more comparable 
estimate of school size across schools with different grade configurations. 

The model. The model was selected after numerous combinations of independent variables were examined. To avoid 
collinearity, some variables were omitted. Other variables were omitted because they did not add substantial 
information to the model. The final model is relatively parsimonious while including the variable of interest (teacher 
credentialing) as well as variables that demonstrated a strong relationship with the dependent variable. All variables are 
significant at the .00 level. 
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Abstract 

Responding to a perceived shortage of school superintendents in Ohio as well as elsewhere in the 
nation, this study examined the conditions of the job that make it attractive or unattractive as a 
career move for principals. The researchers surveyed a random sample of Ohio principals, 
receiving usable responses from 508 of these administrators. Analysis of the data revealed that 
principals perceived the ability to make a difference and the extrinsic motivators (e.g., salary and 
benefits) associated with the superintendency as conditions salient to the decision to pursue such 
a job. Furthermore, they viewed the difficulties associated with the superintendency as extremely 
important. Among these difficulties, the most troubling were: (1) increased burden of 
responsibility for local, state, and federal mandates; (2) need to be accountable for outcomes that 
are beyond an educator’s control; (3) low levels of board support, and (4) excessive pressure to 
perform. The researchers also explored the personal and contextual characteristics that 
predisposed principals to see certain conditions of the superintendency as particularly attractive 
or particularly troublesome. Only two such characteristics, however, proved to be predictive: (1) 
principals with fewer years of teaching experience were more likely than their more experienced 
counterparts to rate the difficulty of the job as important to the decision to pursue a position as 
superintendent, and (2) principals who held cosmopolitan commitments were more likely than 
those who did not hold such commitments to view the salary and benefits associated with the 
superintendency as important. Findings from the study provided some guidance to those policy 
makers who are looking for ways to make the superintendency more attractive as a career move 
for principals. In particular, the study suggested that policy makers should work to design 
incentives that address school leaders’ interest in making a difference at the district level. At the 
same time, they should focus on efforts to reduce the burdens that external mandates contribute 
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to the already burdensome job of school superintendent. 



Introduction 

If popular press coverage is any indication, there seems to be mounting concern about an administrator shortage. 
Anecdotal reports suggest that fewer applicants are now applying for administrative positions than have done so in the 
past (e.g., Cooper, Fusarelli, & Carella, 2000; Pugmire, 1999; Steinberg, 2000). Professional organizations have 
focused in particular on the low numbers of applicants for principalships (NAESP/NASSP, 1998). So far, however, 
there has been little systematic research to clarify the situation by showing how conditions associated with school 
administration, especially the superintendency, relate to educators’ decisions about whether or not to pursue such 
positions. 

Regardless of the extent or severity of the shortage, boards of education have an on-going interest in knowing that 
there will be an ample pool of applicants to fill vacancies (see e.g., McAdams, 1998). And if critical shortages do 
indeed materialize, the concerns of boards will intensify. State policymakers also have an interest because they have 
some control over pipeline issues, such as licensure requirements (see e.g., Ashbaugh & Kasten, 1992; Fenwick & 
Pierce, 2001). Moreover, policymakers bear some responsibility for the conditions that superintendents face on the 
job. For example, in many states, accountability legislation introduces pressure for performance that superintendents 
may find extremely difficult to address (see e.g.. Graves, 1995). 

Despite the efforts of some districts to look for talented leaders from outside of the ranks of the educator workforce 
(e.g., Mathews, 1999), the traditional career path for educational administrators involves the move from teaching to 
the principalship to the superintendency (Glass, 1992). For this reason, the question “what conditions tend to attract 
and what conditions tend to deter principals from considering the superintendency?” seems germane to those 
concerned with the recruitment of capable district leaders. Moreover, among principals, different subgroups might 
find the various conditions associated with the superintendency to be more or less salient to their decision to pursue or 
not to pursue a position as superintendent. 

This study addresses four research questions directly related to these concerns: 

• What conditions associated with the superintendency do principals see as attractive? 

• What conditions associated with the superintendency do principals see as objectionable? 

• What characteristics of principals predispose them to see certain features of the superintendency as attractive 
and certain other features as objectionable? 

• What characteristics of the context in which principals work predispose them to see certain features of the 
superintendency as attractive and certain other features as objectionable? 

Review of Related Literature 

This study fits in with and expands research efforts that have explored the working conditions that characterize school 
leadership positions. In general, this line of inquiry has demonstrated that many educators are reluctant to pursue 
leadership positions because of the demands of the job, the increased pressure to show “results,” and the inadequate 
remuneration (e.g., Cooley & Shen, 2000; Gewertz, 2000; Houston, 1998). 

Recent findings such as these seem to confirm rather than to contradict findings from earlier studies of the 
superintendency. Raymond Callahan (1962), for example, provided considerable evidence suggesting that, even in the 
early 1900's, superintendents (especially those in large cities) were pressured to demonstrate accountability both in 
terms of financial management and in terms of educational outcomes. Although there have been challenges to 
Callahan’s claim that superintendents were extremely vulnerable as a result of these pressures for “scientific 
management” of schools (see e.g.. Button, 1991 ; Eaton, 1990; Thomas & Moran, 1992), most educational historians 
acknowledge that such pressures did exist (see e.g., Cuban, 1976). 

Contemporary case studies (e.g., Johnson, 1996) also demonstrate the complexity of the role that superintendents 
undertake when they try to balance educational, managerial, and political leadership in ways that promote school 
improvement. According to some researchers (e.g.. Glass, Bjork, & Brunner, 2000), the complexities confronting 
superintendents have increased in recent decades, compounding the pressures traditionally associated with the role. 
Several conditions account for the added pressure. First, state-level requirements — for instance, for school and district 
accountability ~ have intensified. Because, in many communities, local citizens do not concur with the state education 
agency’s interpretation of educational quality, such mandates often sandwich superintendents between the interests of 
their constituencies and the interests of the state (see e.g., Chalker, 1 999). Another source of pressure results from the 
increasing power of teachers’ unions (Haley & McDonald, 1988). Interactions with these groups can become 
particularly troublesome when union interests do not fit in well with the school reform efforts desired by district 




leaders (see e.g., Ballou, 1999; Lieberman, 1984; but cf. Koppich, 1991). Finally, changing demographics make the 
job of school administrators more complex, as various community groups compete to define the mission of schools in 
ways that match their values and expectations (see e.g., Houston, 1998; Portin, 1997). 

Superintendents’ jobs are also made more difficult when these school leaders are unable to gamer adequate resources 
to implement the sorts of district improvements expected of them (Houston, 1998). According to Houston (2001), the 
expectations for reform and the resources allocated to districts are out of alignment. In fact. Glass and associates 
(2000) found that superintendents identify lack of financial resources as the one factor that most seriously limits their 
effectiveness. Moreover, in districts with limited resources, superintendents’ low salaries may provide these 
administrators with another source of job-related stress (Yvarra & Gomez, 1995). 

In spite of the difficulties of district leadership, research clearly shows that most superintendents are satisfied with 
their jobs. In a survey of superintendents from several different states. Cooper and associates (2000) found that most 
of these school administrators reported that their jobs were challenging, rewarding, and satisfying. In addition, these 
superintendents overwhelmingly regarded themselves as effective, with 96% of those surveyed agreeing that their 
work made a significant difference in children’s lives. Similar findings were reported by Ramirez and Guzman in 
their study of rural superintendents in Colorado. Hill and Ragland (1995), moreover, found that long work hours did 
not seem to detract from superintendents’ job satisfaction. It appears that the ability to make a difference and to 
exercise leadership may offer sufficient satisfaction to superintendents to enable these school leaders to persist in their 
work despite its obvious challenges (Wesson & Grady, 1993; 1994). 

Methods 

We surveyed a random sample of 826 of the 3644 principals in the state of Ohio (i.e., a sample draw with a 95% 
confidence level and 3 confidence interval) using an instrument that included 19 variables related to conditions of the 
superintendency. Each respondent was asked to rate on a 4-point Likert scale the extent to which a specific condition 
would affect his or her decision to pursue a position as superintendent. The variables were organized into three scales 
reflecting the types of concerns that, based on previous research, seemed to be salient. Confirmatory factor analysis 
demonstrated that these three scales - the “making a difference scale,” the “hard job scale,” and the “extrinsic 
motivator” scale - were, in fact, discrete and explanatory. 



The instrument also included questions eliciting demographic information about respondents (i.e., age, gender, years 
as a teacher, years as an administrator, highest degree obtained, experience as a coach). In addition a scale including 
six items measured the localist and cosmopolitan commitments of the principals. Localists were those who believed it 
was most important to remain in their current districts, to live close to where they were bom and raised, and to stay in 
the same communities for most of their lives. Cosmopolitans were those who believed it was most important to make 
a name for themselves in the field of education, travel to broaden their horizons, and leave home in order to seek 
career opportunities. This construct was deemed important because of the pioneering but somewhat neglected work of 
Carlson (1972), suggesting that place-bound (i.e., localist) and career-bound (i.e., cosmopolitan) superintendents 
harbor different reasons for pursuing leadership positions and follow different career trajectories. 

In addition to data collected via the instrument, we imputed contextual data from two other sources: the Ohio 
Department of Education’s Educational Management Information System (EMIS) and the National Center for 
Education Statistics’ Common Core of Data. By using these publicly accessible resources, we were able to add to our 
data set accurate information about the community contexts in which our responding principals worked. The variables 
most salient to our analyses included locale (rural, non-rural), Appalachian/non-Appalachian, school SES (measured 
as percent eligible for free or reduced lunch), school size, and total per pupil expenditure. 



Descriptive statistics were computed for each variable; then data were analyzed to determine (1) the extent to which 
the three sets of conditions - making a difference, hard job, and extrinsic motivation - were salient to principals in 
their decision-making regarding pursuit of a superintendency, (2) the characteristics of principals that predicted the 
extent of their concern about each of the three sets of conditions, and (3) the features of school context that predicted 
the extent of principals’ concerns about each of the three sets of conditions. 

Findings 

We received responses from 508 principals - a response rate of approximately 62%. Of the respondents, 36% were 
female and 64% male. Their average age was 47.3 years. The average years of experience as a teacher was 12.8, and 
the average years of experience as a principal was 10.2. In addition, 58.6 % of respondents had worked as coaches. 
Furthermore, among these principals, 51 .8% tended to be more cosmopolitan than localist, while 48.2% tended to be 
more localist than cosmopolitan. With regard to highest degree earned, .6% held the Bachelor’s, 88.2% held the 
Master’s, 3.2% held the specialist degree, and 8% held the doctorate. 

Among the principals, moreover, 24.1% worked in rural schools and 12.8 % worked in schools within Appalachian 
counties (as identified by the Appalachian Regional Commission). Schools’ sizes, SES, and levels of funding, of 
course, varied considerably across the sample. 
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Preliminary descriptive analyses showed the individual variables that were most salient to principals’ decision to 
pursue the job of superintendent. These variables were classified intuitively as “appealing conditions” and 
“unappealing conditions;” and the strength of each was revealed in its mean rating by the principals. Table 1 presents 
descriptive statistics for each “appealing” and each “unappealing” condition. As these data indicate, principals found 
the following four conditions most appealing: the chance to have a greater impact, the anticipated satisfaction 
associated with “making a difference,” the opportunity to implement creative personal ideas, and the anticipated 
satisfaction associated with the ability to provide support to school and district staff. They found the following four 
conditions least appealing: increased burden of responsibility for local, state, and federal mandates; the need to be 
accountable for outcomes that are beyond an educator’s control; low levels of board support; and excessive pressure 
to perform. 



Table 1 

Principals Ratings on a 4-point Likert Scale of Appealing and Unappealing Conditions of the 

Superintendency 



Appealing Conditions 


M 


SD 


N 


chance to have a greater impact 


3.17 


.80 


466 


anticipated satisfaction associated with “making a difference” 


3.11 


.84 


467 


opportunity to implement creative personal ideas 


3.06 


.77 


463 


anticipated satisfaction associated with the ability to provide support to school and district staff 


3.05 


.75 


462 


high levels of board support 


2.93 


.97 


454 


improved annual salary 


2.78 


.85 


466 


improved benefit package 


2.77 


.88 


468 


greater control over work schedule 


2.72 


.83 


464 


increased opportunities for professional growth 


2.70 


.86 


467 


higher status 


2.42 


.84 


465 


Unappealing Conditions 




increased burden of responsibility for local, state, and federal mandates 


3.08 


.93 


467 


need to be accountable for outcomes that are beyond an educator’s control 


2.94 


.93 


465 


low levels of board support 


2.90 


1.03 


457 


excessive pressure to perform 


2.90 


.96 


465 


stress associated with anticipated conflict with teachers’ unions 


2.75 


.97 


463 


increased work load 


2.64 


.95 


464 


lack of clarity about job expectations 


2.45 


.87 


466 


need for greater amounts of technical knowledge 


2.32 


.8 


466 


superintendency is overly dominated by males 


1.78 


.90 


464 



As indicated in the discussion of research methods above, we made the assumption, based on our reading of the 
related literature, that several of the variables identifying appealing and unappealing conditions would combine to 
form discrete and meaningful scales. We tested our assumptions about the items that would be associated by 
performing a confirmatory factor analysis in which we used varimax rotation to accentuate strong associations. This 
analysis showed that the significant factors comprised of associated items explained 50.53% of the variance on the 
instrument and corresponded to three themes that were clearly evident in previous literature. These themes related to 
(1) the satisfaction associated with “making a difference,” (2) the distress associated with the difficulty of the job 
(the”hard job” factor), and (3) the satisfaction associated with extrinsic rewards such as salary and benefits (the 
“extrinsic motivators” factor). Appendix A presents the items that load on each of the significant factors and their 
factor loadings. We identified factors as reliable using Stevens (1996) criteria. In order to examine the extent to which 
the three sets of concerns represented by the three reliable factors were salient to the principals, we computed and 
compared scale means using paired-sample t-tests. We found that principals rated “making a difference” as most 
salient (mean = 3.02), “hard job” as second most salient (mean = 2.82), and “extrinsic motivators” as least salient 
(mean = 2.66). Differences between pairs of means were all highly significant (p < .0001). 



We then constructed multiple regression equations to identify significant predictors of level of concern for each of the 
three sets of conditions. In each equation we included the scale measuring a set of conditions (i.e., “making a 
difference,” “hard job,” or “extrinsic motivators”) as the dependent variable and the characteristics of principals or of 
their schools as independent variables. In the equations that considered the influence of the characteristics of 
principals we excluded “highest degree obtained” from among the independent variables because, with over 88% of 
respondents holding the Master’s as the highest degree, there was very little variance. We also excluded the 
independent variable “age” because of its moderate bivariate correlation with “years of experience as a principal” (r 
= .52). In the equations that considered the principals’ school contexts, we omitted the dummy variable, 
“Appalachian/non-Appalachian” because of its bivariate correlation (r = .32) with the variable, rural/non-rural. 

With regard to the effect of principals’ characteristics on the extent to which they saw “making a difference” as 
salient, the overall equation was non-significant and explained a minute fraction of the variance. (See Table 2.) It 
appears that the characteristics of principals we measured had little bearing on the extent to which they saw the 
possibility of making a difference as important to their decision to pursue or not to pursue the position of 
superintendent. 



Table 2 

Summary of Multiple Regression Analysis of Principal Characteristic Variables Predicting Concern 

for “Making a Difference” (N = 410) 



Variable 


B 


SE(B) 


p 


Gender 


.108 


.081 


.081 


Years as teacher 


.003 


.005 


.036 


Years as principal 


-.003 


.005 


-.033 


Experience as a coach 


.011 


.075 


.009 


Localism 


-.023 


.032 


-.036 


Cosmopolitanism 


.088 


.031 


.141 


Adjusted = .0 1 , /? = . 1 1 9 



Results were similar for the “hard job” scale, where the overall equation was significant (p = .048) but explained very 
little of the variance on the scale (adjusted R square = .017). (See Table 3.) Only one variable, years as a teacher, had 
a significant effect. Principals with less teaching experience were more likely than their more seasoned counterparts to 
rate the difficulty of the job as salient to the decision to pursue a position as superintendent. 

Table 3 

Summary of Multiple Regression Analysis of Principal Characteristic Variables Predicting Concern 

for “Hard Job” (N = 396) 



Variable 


B 


SE(B) 


p 


Gender 


-.138 


.085 


-.100 


Years as teacher 


-.011 


.006 


-.109* 


Years as principal 


-.001 


.005 


-.014 


Experience as a coach 


-.039 


.079 


-.029 


Localism 


.066 


.034 


.097 


Cosmopolitanism 


.021 


.033 


.033 


Adjusted = .017,/? = .048, *p < .05 



With regard to the “extrinsic motivator” scale, the overall equation was significant but also explained relatively little 
(3.7%) of the variance on the scale. (See Table 4.) One predictor, cosmopolitanism, exerted a significant influence. A 
principal was more likely to view the salary and benefits associated with the superintendency as important if he or she 
held cosmopolitan commitments. 



Table 4 

Summary of Multiple Regression Analysis of Principal Characteristic Variables Predicting Concern 

for “Extrin^c^^tivator” (N = 415) 



Variable 


B 


SE(B) 


p 


Gender 


.137 


.080 


.101 


Years as teacher 


-.001 


.005 


-.012 


Years as principal 


-.002 


.005 


-.016 


Experience as a coach 


.076 


.075 


.058 


Localism 


-.013 


.032 


-.020 


Cosmopolitanism 


.129 


.032 


.200* 


Adjusted = .037 , p = .002, p < .05 



The influence of school context features on the strength of principals’ concern for the three major conditions of the 
superintendency (i.e., “making a difference,” “hard job,” and “extrinsic motivators”) was even less pronounced than 
the influence of principal characteristics. None of the equations predicting the strength of principals’ concern for these 
conditions was significant. Summary statistics for these regression models are provided in Tables 5, 6, and 7. 

Table 5 

Summary of Multiple Regression Analysis of School Context Variables Predicting Concern for 

“Making a Difference” (N = 382) 



Variable 


B 


SE(B) 


p 


Locale 


.065 


.081 


.044 


Total per pupil expenditure 


.000 


.000 


.072 


School SES 


-.000 


.001 


-.019 


School size 


-.000 


.000 


-.022 


Adjusted = -.04, p = .675 



Table 6 

Summary of Multiple Regression Analysis of School Context Variables Predicting Concern for 

“Hard Job” (N = 364) 



Variable 


B 


SE(B) 


p 


Locale 


-.072 


.086 


-.046 


Total per pupil expenditure 


.000 


.000 


.060 


School SES 


.000 


.001 


-.117 


School size 


.000 


.000 


-.085 


Adjusted = .007, p = . 1 64 



Table 7 

Summary of Multiple Regression Analysis of School Context Variables Predicting Concern for 

“Extrinsic Motivator” (N = 384) 



Variable 


B 


SE(B) 


p 


Locale 


Ml 


.080 


.086 


Total per pupil expenditure 


.000 


.000 


.081 


School SES 


.000 


.001 


-.031 
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School size | >000 | .000 | -.001 

Adjusted = .001,p = ,361 



Discussion 

Overall, the analyses showed that principals rated the ability to make a difference as a superintendent as the most 
compelling reason guiding their thinking about whether or not to pursue such a position. Their concern about making 
a difference was reflected in their high ratings on questionnaire items related to the superintendents* role in providing 
support to school and district staff, the ability of superintendents to implement creative personal ideas, and the general 
impact that district leaders can have. Based on this study, it seems, principals’ perspectives correspond closely to 
those of practicing superintendents with respect to the features of district leadership that are perceived to be most 
compelling (e,g., Cooper et al,, 2000; Houston, 2001; Wesson & Grady, 1994), 



This finding has important ramifications for policy and practice. Regarding professional preparation, those who 
design university and workshop programs for aspiring superintendents might find it useful to focus on the 
competencies that enable school leaders to promote district-level improvement. Giving administrators tools that can 
help them make a difference builds on these educators’ intrinsic motives for pursuing leadership roles (cf , Lortie, 
1975), Furthermore, local boards would be well served by creating conditions that support superintendents’ efforts to 
foster meaningful district-level change. Increasing a superintendent’s term of contract, for example, might give him or 
her sufficient chance to have a noticeable impact on the district’s performance (cf Yee & Cuban, 1996), 



Our analyses also revealed that principals were concerned about the challenges of the superintendency. Among the 
variables included on the “hard job” scale, they rated the following as most salient: “superintendent’s increased 
burden of responsibility for local, state, and federal mandates” and “the need to be accountable for outcomes that are 
beyond an educator’s control,” These responses suggest that the current focus on accountability may be adding to the 
stresses already associated with the superintendency (see e,g,, Cooley & Shen, 2000), Policies that promote 
accountability mechanisms responsive to local rather than state concerns may temper such added stress (e,g,, 

Mathews, 1996), 

Principals with fewer years as teachers were more concerned than others about the difficulty of the superintendency, 
and this finding suggests particular cautions regarding districts’ recruitment of principals. Specifically, districts may 
want to avoid hiring as principals applicants who have limited experience as teachers. This suggestion, of course, also 
corresponds to recommendations concerning the background necessary for instructional leadership (Miller, 1 987), 
and it fits in with certain research findings about predictors of effective school administration (Ballou & Podgursky, 

1 995), Our study, however, provides tentative support for the practice of hiring experienced teachers as principals on 
the grounds that these individuals will be more likely than their less experienced counterparts to pursue a full career in 
administration, eventually assuming the chief executive role. In times of administrator shortages, of course, districts 
with few other options will be likely to offer principalships to relatively inexperienced educators. This practice may 
enable such districts to fill school vacancies but may limit their long-term efficacy in cultivating leadership at the 
district level. 

Our study also showed that extrinsic motivators such as salary, benefits, control over work schedule, and status were 
also important considerations when principals thought about the possibility of applying for positions as 
superintendents. In fact, principals who were committed to cosmopolitan values seemed especially attuned to these 
conditions. This finding is not surprising considering that these individuals place priority on accomplishment of career 
goals. For these career-bound administrators, work in small, lower-paying districts may often serve as stepping-stones 
to larger, more prestigious roles (see e,g., Carlson, 1972), Moreover, this finding has important practical 
consequences since, at least in Ohio, more than half of all principals harbor stronger cosmopolitan than localist 
commitments. 



These results suggest that local boards and state policy makers should work to find ways to create incentive packages 
that are attractive to aspiring superintendents. According to several commentators, such compensation packages need 
to address salary, portable retirement plans, annuities, insurance, tuition reimbursement, expense account allowances, 
and support for moving expenses (sec c,g,. Educational Research Ser\'ices, 1990; Heller, 1991: Shannon, 1987), 

One other finding from this study, namely the uniformity of principals’ concerns across demographic differences, 
seems pertinent. As our regression equations revealed, just a few personal characteristics and no school context 
characteristics exerted a significant influence on the strength of principals’ concern for the three sets of conditions 
associated with the work of superintendents. This finding suggests that principals’ views of the conditions of 
administrative work may be shaped by forces other than those attached to conventional social categories. Principals’ 
views, it seems, are formed in an ideological space that transcends social location. 

This conclusion leads to speculation about the ways professional socialization may function to define not only the 
character of school administrators’ work but also their interpretations of its scope and meaning. And such speculations 
provide a hopeful path back to the profession itself as a place to look for continued, perhaps revitalized, support for 
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the superintendency. This analysis does not go so far as to espouse a laissez-faire response to the problem of 
superintendent shortages (i.e., “if you advertise it, they will come”), but it does suggest that the profession itself, 
without much mediation from local and state policy makers, may be able to reinvest the role of superintendent with 
sufficient authority and efficacy to once again make its attainment the aspiration of those educators with the greatest 
talent for leadership. 
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Appendix A 

Variable Loadings > .40 on the Three Significant Factors 
(Principal Components Analysis with Varimax Rotation; N = 417) 



Factor and Variables 


Factor Loadings 


% Variance Explained 


Making a Difference 




20.7 


Chance to have greater impact 


.83 




Making a difference 


.77 




Opportunity to implement ideas 


.76 




Provide support to staff 


.74 




Opportunities for growth 


.60 




Hard Job 




17.28 


Responsibility for mandates 


.80 




Accountability for outcomes 


.76 




Increased work load 


.70 




Conflict with unions 


.70 




Low board support 


.69 




Excessive pressure 


.65 




Unclear job expectations 


.56 




High board support 


.55 
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Extrinsic Motivators 




12.55 


Improved salary 


.83 




Improved benefits 


.79 




Control over work schedule 


.65 




Higher status 


.55 
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Abstract 

This article assesses a socio-constructivist model for training K-12 teachers in Brazil, in the use 
of Informatics in education. The method applied combines both face-to-face exchanges and a 
WEB-based distance approach made possible by Internet technology. The characteristics of such 
training and its main objectives are analyzed according to the collected data presented. A 
descriptive single case study research methodology is applied. The main conclusions reached by 
this research — based on the use of a systemic framework — are presented, mainly those 
addressing the importance of care and coherence for knowledge creation in a socio-constructivist 
training model developed with the help of the Internet. Comparisons between this model and the 
traditional one are also presented. 



Introduction 

In 1997, the Brazilian Ministry of Education and Culture (MEC) defined and launched a national public policy for the use of 
Informatics in Brazilian K-12 public schools, by means of a broad-based Program involving all 27 Brazilian States, known as 
PROINFO. While the Federal Government is in charge of investing in equipment and teacher training, the Brazilian states 
have autonomy to implement their Projects of Informatics in Education, as they think fit. 

PROINFO, The National Program of Informatics in Education, aims to initiate the public educational system into the process 
of using state-of-art technology in the classroom, as well as to train human resources to develop their work adequately. In 
order to accomplish this target, PROINFO along with the acquisition and deployment of the necessary equipment, has stressed 
the need for training the teachers, using partnerships with universities, as training agents. Hence, together with state and 
municipal governments, NTEs {Nucleos de Tecnologia Educacional) were created to work as hubs of teacher training. NTEs 
are decentralized structures to promote the dissemination of Informatics within the pedagogical practice in public schools in 
each state. In these hubs, there are multipliers (teacher trainers) with qualifications (more than 360 course hours) in 
Informatics in Education given by a university involved in the program. 

The greatest hurdle to the introduction of Informatics into the educational system has been the development of qualified 
human resources to keep abreast of the rapid transformations inherent to this area, along with the ever-present need to improve 
the quality of public education. 





Taking part in this project, Rio Grande do Sul State, the southernmost Brazilian State, near the border with Argentina and 
Uruguay, has created 12 NTEs across the state in which 57 multipliers (teacher trainers) are already working. In Rio Grande 
do Sul, the State Commission in charge of implementing PROINFO across the state invited the Federal University of Rio 
Grande do Sul - UFRGS (www.ufrgs.br) to be its partner in this endeavor. Thus, the university developed the first training 
program for 57 multipliers, based on a face-to-face approach. Due to the success of this course, the State asked UFRGS to 
stage a second course in order to train more multipliers. 



Research has been undertaken by the LEC (Cognitive Studies Laboratory) of UFRGS (www.psico.ufrgs.br/lec/) in the ongoing 
use of instructional technology and its role in the development of innovative educational practices. Consequently, new training 
methodology was developed by LEC to train the multipliers, combining face-to-face tuition with Web-based Instruction in 
order to be used for distance training. 



This second training program started in July 1999 and ended in October 1999, involving 29 teachers from Rio do Grande do 
Sul State, who are expected to work as multipliers in the NTEs across the state. 

Impetus for Telematics-Based Training 

The impetus for a new training model for teacher training was based on the outcome of research developed by UFRGS in the 
realm of Informatics in Education. This has already been translated into theoretical models leading to new methodologies to 
orient changes in classroom practice (Projeto Educadi — http://educadi.psico.ufrgs.br/cursos/index.html), combined with the 
university’s experience in the application of these models during the first specialization course on Informatics in Education 
given to the NTE multipliers of the PROINFO Program. Hence, it was concluded that it would be possible to develop and 
implement a second course, based on both a face-to-face approach and Internet-based distance training, the theoretical 
background of which was based on the development of interdisciplinary projects using Telematics. 



The great challenge is to make the teachers of the public system aware of the need for incorporating new information 
technology in their daily practice in the classroom. The use of the technology demanded by the course facilitates the creation 
of learning and practice communities (Wenger & Snyder, 2000) involving both students and experts, as well as the exploration 
of the computer environment by both students and teachers, leading to enhanced socio-cognitive development and a shared, 
collective pooling of knowledge. 

It was expected from the positive results of the first course, that this second course would permit the elimination of learning 
verticality and that each teacher and student of the course, would develop his/her own learning using a shared construction 
model, leading to the accomplishment of autonomous knowledge. Hence, the course was geared to stress the development of 
projects by small groups, addressing three aspects ( www.psico.ufrgs.br/mec-nte2/): 

1 . the student, seeking new technological resources in order to create knowledge leading to problem-solving related to all 
knowledge areas addressed in the course; 

2. the teacher interacting with other teachers and with students within computerized environments in the public 
educational system, trying to grasp the established socio-cognitive relationships, as well as the participative, albeit 
autonomous knowledge-creation process; 

3. the tutor in charge of training new human resources in his/her place, attempting to understand the interdependency 
relationships and, at the same time, the autonomous networks created by his/her peers, as the verticality of the system 
is eliminated and actual cooperative and truthful relationships based on reciprocity are developed. 

Operational Training Targets 

The training under scrutiny here had the following operational targets: (www.psico.ufrgs.br/mec-nte2/) 



• To train the teacher trainers involved in PROINFO, through a course on Informatics in Education lasting more than 
360 hours (a lato-sensu graduate course, in Brazil), in order to qualify them to be multipliers at the NTEs throughout 
Rio do Grande do Sul State; 

• To develop this 440-hour course in such a way that 320 hours of it are based on a face-to-face approach in Porto 
Alegre (Rio Grande do Sul State Capital), the remaining 120 hours to be dedicated to a distance training model, 
through the Internet; 

• To take advantage of the 120 hours allocated to Distance Training, implementing the proposal presented in the Rio 
Grande do Sul State Project on Informatics in Education, in the schools assisted by each NTE, the teachers of which 
are attending the course. 

Course Description 



ERIC 



Course Methodology, The methodology of the course is divided into three main themes: face-to-face; mediated interaction and 
distance based instruction, as explained below: 
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• face-to-face: at this stage there are thematic workshops and seminars to plan and develop the interdisciplinary and 
diversified projects. 

• mediated interaction: at this stage the simultaneous use of Internet services is intensified, namely: electronic mail, 
discussion lists, forums, real-time communication (IRC, Chat, MOOs, CuSeeMe etc.) and, concurrently a database for 
storing products like reports addressing the progress of each participant, texts, and on-going assessment portfolios is 
created, all of which are generated by the students during the course. Hence, the student activity records and data 
collection of their production are stored. This material is published in a site located in the Internet server of the course, 
along with the teachers and students’ home pages. 

• distance-based instruction: at this final stage an interdisciplinary project is developed by groups of students, who are 
already on-site at their NTEs, by giving training to other teachers and receiving orientation through the Internet 
regarding their dissertation 

Course Structure. The course is divided into two stages, as presented below: 

• First Stage - Face-to-Face (320 hours) 

The first stage is compounded of two modules {Module 1 and 2) over eight weeks, consisting of 40 hours per week. A 
section known as Autonomous Production Time ~ AFT^ (8 hours per week incorporated in the 40-hour week) is 
programmed into this stage. 

• Autonomous Production Time - TP A (64 hours/activity) 

This time is to be used by the students in activities such as individual or collective studies, interviews with experts 
whose expertise is addressed in the project being developed, experiments with teachers and pupils in the classrooms, 
tracking of schools which are using ICT to develop pedagogical projects, finding solutions to challenges in the 
Informatics arena, information-searching on the Internet, data and information exchange through the Internet, 
instrumental software training, to name but a few. 

All the communications, discussions and reflections, as well as daily and final reports, are made available in a 
common area for the entire group under training, so as to permit interaction and exchange among the participants. 

• Module / (132 hours/activity) 

Structured into thematic workshops to be developed according to the group interests and the problems encountered by 
them. Each group (of 3 members) takes part in the thematic workshops, which are developed on a project basis 
approach. 

The first project (40 hours/activity) focuses on problem-solving in order to establish relationships between the 
different subjects of the curriculum. In this project the teachers are considered pupils addressing the basic level 
curriculum, formulating and solving problems, elaborating reports and evaluating their experiences. 

The second project (80 hours/activity) addresses the understanding and analysis of the socio-cognitive and emotional 
relationships between teachers and students, and evaluates their ongoing knowledge acquisition in ICT-based learning 



The third project lasts 12 hours on a face-to-face basis with 120 hours of distance learning. The face-to-face stage 
focuses on the elaboration of the teacher-training project for use of ICT in the classroom. This project is developed by 
each one of the NTEs involved. The second stage is developed on a distance-learning basis, focusing on 
implementation of the project. 

• Module 2 ( 1 24 hours/activity) 

Consists of three Basic Seminars: “New Information and Communication Technologies for Learning Environments”; 
“Theoretical Basis for Learning in Computerized Environments”; and “Methodologies of Educational Interaction and 
Intervention in Computerized Learning Environments”. 

• Second Stage - Distance-Based ( 120 hours) 

This stage is held in the students’ original NTEs. Each NTE (group of 6 students) develops an integrated project, 
lasting one and a half-months, being both theoretical and practical, based on their experiences with the students on the 
course and on their experience with the students of the schools belonging to each NTE. Geared towards motivating the 
students of the Basic Level to build relevant knowledge in computerized environments, the teachers involved in this 
experience are supported in the cooperative construction of relevant knowledge in order to assist the students in the 
computerized environment. They are also assisted in the construction of sharing methodology for interaction and 
intervention. They are encouraged to use the available ICT as enablers in this process. In this stage, the mentors of the 



environments. 




course assist the students of the course on a distance-basis approach, via the Internet. 

Research Design 

Assessment generally has at least one of the following three purposes: to improve, to inform, and/or to verify. The aims of this 
assessment are to provide information that can be used to determine whether or not intended outcomes are being achieved and 
how the project can be improved. In addition, the assessment process was structured to inform decision-makers about relevant 
issues that affect the project. (Rogers & Sando, 1996) 

In this assessment process, it is important to distinguish between “formative” and “summative” assessment. Formative 
assessment is the collection of data and feedback of results on an ongoing basis. Formative assessment is designed to provide 
information for the purpose of improving the project or process being assessed. (Rogers & Sando, 1996). Summative 
assessment is designed to produce information that can be used to make decisions about the overall success of the project or 
process. Hence, this is a summative assessment. 

Methodology 

The research methodology used in this article was the single case studyone. Case study is particularly suitable to answering 
“how” and “why”questions and are well suited to generating and building theory in an area where little data or theory exists 
(Yin, 1994). It also enables the researcher to use “controlled opportunism” to respond flexibly to new discoveries made while 
collecting new data (Eisenhardt, 1994). Yin’s tactics (construct validity; internal validity; external validity; and reliability) 
were carefully considered in this research. 



In particular, construct validity was dealt with in the study through the use of multiple sources of evidence, the establishment 
of a chain of evidence, and having the members of the group reviewing the draft case study report. Internal validity issue is not 
necessary for descriptive case studies (Yin, 1994) and external validity in the findings is taken into account, mainly by 
applying replication logic. Finally, the reliability of the results was taken for granted by using a case study protocol and 
developing a case study database. 



To conduct this research, some actions were undertaken and a theoretical framework was used. Several steps were conducted, 
in order to develop a working methodology to be applied in this research task. These steps are listed below: 



• The project is presented in detail as conceived by its creators. 

• The modus-operandi of the Program, i.e., its structure, is presented, so as to make clear how the outcomes are intended 
to be accomplished. 

• Data to be analyzed are collected from different sources. In this case the following data were collected: 

o Data and Information about training, available on an Internet site at the following URL: 

(www.psi CO. ufrgs.br/mec-nte2); 

o The rationale of the project taken from academic papers developed to depict the program itself (Nevado et ai, 
1999); 

o Student portfolios and projects deployed on the Internet also made available - although in a reserved area - 
through permission granted by the training coordinators; 

• Analysis of the outcomes of the project obtained from the application and consolidation of questionnaires and 
interviews with both the students and teachers involved; 

• Classroom observations and video-recordings, to analyze whether the teachers trained in a constructivist environment 
are applying this approach to train their students - the other teachers of the public education system; 

• Comparison between the forecast and actual accomplished objectives so as to evaluate the program; 

• Analysis of the cost-effectiveness of the Program; 

• Conclusions and analysis of the strengths and weaknesses of the training program. 

Theoretical Framework 

The questionnaire and direct observation of the training, mainly the WEB-Based Instruction stage were drawn primarily from 
the paper: “Effective Dimensions of Interactive Learning on the World Wide Web”, Reeves & Reeves (1997). This model has 
applications in research, implementation and assessment of Web-based Instruction Programs such as this project. 

The proposed model includes ten dimensions of interactive learning on the World Wide Web: (1) pedagogical philosophy, (2) 
learning theory, (3) goal orientation, (4) task orientation, (5) source of motivation, (6) teacher role, (7) metacognitive support, 
(8) collaborative learning, (9) cultural sensitivity, and (10) structural flexibility. 



Each of the ten dimensions in this model is presented as a two-ended continuum with contrasting values at either end. Needless 
to say, the world is rarely dichotomous and there is more complexity involved in learning than any of these dimensions 
represent. However, the individual dimensions themselves are not as important as the interplay among the ten dimensions that 
represent the instructional designs of various WBI sites. 



1 . Pedagogical Philosophy (In struct! vist <=> Constructivist) 

The debate between instruct! vist and constructivist approaches to teaching and learning continues through education 
and training (Kafai & Resnick, 1996). Instructivists stress the importance of objectives that exit apart from the learner. 
Little emphasis is placed on learners per se, who are viewed as passive recipients of instructions or treated as empty 
vessels to be filled with learning. By contrast, constructivists emphasize the primacy of the learner’s intentions, 
experience and cognitive strategies. According to constructivists, learners construct different cognitive structures 
based upon their previous knowledge and what they experience in different learning environments. It is paramount for 
constructivists that learning environments be as rich and diverse as possible. Instead of an empty vessel, the learner is 
regarded as an individual replete with pre-existing motivations, experiences, aptitudes and knowledge. Tasks to be 
accomplished and problems to be solved must have personal relevance to the learner. The constructivists believe that 
what we know is constructed - both individually and socially - based on prior experience. 

2. Learning Theory (Behavioral <=> Cognitive) 

According to behaviorists, the critical factor in learning is observable behavior, and instruction involves shaping 
desirable behaviors through the arrangement of stimuli, responses, feedback, and reinforcement. A stimulus is 
provided (e.g. a short presentation of content), then a response is elicited, often via a question. Feedback is given as to 
the accuracy of the response, and positive reinforcement is given for accurate responses. Inaccurate responses result in 
a repetition of the original stimulus, and the cycle begins again. Cognitive psychologists place more emphasis on 
internal mental states than on behavior. A cognitive taxonomy of internal learning states includes simple propositions, 
schema, rules, skills, mental models and so forth. They claim that a variety of strategies - including memorization, 
direct instruction, deduction, drill and practice, and induction, are required in any learning environment, depending 
upon the type of knowledge to be created by the learner. 

3. Goal Orientation (Sharp <=> Broad) 

The goals for education and training can range from sharply focused ones to general higher-order ones. Hence, the 
goal orientation of WBI systems varies in degree of focus from sharp to broad (Cole, 1 992). 

4. Task Orientation (Academic <=> Authentic) 

The context of learning is enormously important to adults (Merriam, 1993). An academic design would depend 
heavily on having the learners carry out traditional academic exercises. By contrast, an authentic design would engage 
the adults in practical activities such as preparing job applications, thereby situating practice and feedback within 
realistic scenarios. If knowledge, skills, and attitudes are learned in a practical context, they will be used in that and 
similar contexts. 




5. Source of Motivation (Extrinsic <=> Intrinsic) 

Motivation is a primary factor in any theory or model of learning (Amabile, 1 997). Every new educational technology 
promises to be intrinsically motivating. This dimension ranges from extrinsic (i.e., outside the learning environment) 
to intrinsic (i.e., integral to the learning environment). Intrinsically, motivation instruction is elusive regardless of the 
delivery systems. 

6. Teacher Role (Didactic <=> Facilitative) 

The teacher role continuum ranges from didactic to facilitative. In the former role, the teacher presents information 
and asks learners to memorize information and recall it later in tests. The latter role assigns cognitive responsibility to 
the learners, as they should be responsible for recognizing and judging patterns of information, organizing data, 
constructing alternative perspectives, and presenting new knowledge in meaningful ways, wherein the teachers are 
mentors and tutors of this process. 

7. Metacognitive Support (Unsupported <=> Integrated) 

Metacognition refers to a learner’s awareness of objectives, ability to plan and evaluate learning strategies, and 
capacity to monitor progress and adjust learning behavior to accommodate needs (Flavell, 1979). The metacognitive 
support dimension is unsupported at one end of the continuum and integrated at the other. Recapitulation of the 
students’ strategies at any point in the problem-solving process, as well as construction of Web-based portfolios are 
examples of how support for reflection and metacognition might be provided in WBI. 

8. Collaborative Learning Strategies (Unsupported <=> Integral) 

The Collaborative Learning dimension ranges from a complete lack of support for collaboration to the inclusion of 
collaborative learning as an integral feature. Cooperative and collaborative learning refers to instructional methods in 
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which learners work together in pairs or small groups to accomplish shared goals. 

9. Cultural Sensitivity (Insensitive <=> Respectful) 

All instructional systems have cultural implications. In an insensitive approach the training is developed regardless of 
the culture and diversity of the learners it is intended to address. On the other hand, a respectful approach should be 
based on the diversity in the populations where the system will be used so that the overall learning environment is 
enhanced. It is unlikely that WBI Training can be designed to adapt to every cultural norm, but sites should be 
designed to be as culturally sensitive as possible, 

10. Structural Flexibility (Fixed <=> Open) 



“Fixed” systems, still dominant in education, are usually limited to specific places, e.g,, a classroom or laboratory, at 
specific times, e.g., 50 minutes class period. Independent of time and/or location constraints, the learner can use 
“Open” systems. The World Wide Web provides opportunities for more asynchronous (open) learning, although some 
Web-based learning tools are temporally fixed (synchronous), such as chats, video-conferences, MOOs and MUDs. 

The research aims to have Table 1 completed by the learners so as to evaluate the program’s outcomes. 

Table 1 
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Data Survey and Analysis 

Theoretical Model of Training applied by the LEC 



The LEC (Cognitive Studies Laboratory) has published several papers addressing a socio- constructivist approach to train 
teachers using Information and Communication Technologies as enablers. According to LEC (Nevado et ai, 1999) the answer 
to the question: “If new practices are recommended to the teachers under training, why do the trainers themselves not apply 
this approach in their own classes?” lies at the foundation of a theoretical frame of reference upon which this practice is based. 



So, to design a new pedagogical model for computerized environments it is necessary to create a theoretical framework which 
leads the teachers to understand more fully the affective and socio-cognitive processes that are developed when students are 
interacting with Information and Communication Technologies. This theoretical approach is even more necessary considering 
that the digital media increases the interactive processes among the players involved (teachers, students, experts, community). 
This model can assist in developing interactive models for teacher training. 

According to Costa et al. (1997), the central idea of the theory, the basis of the model, is presented below in Figure 1 . 
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Figure 1. Theoretical Framework for Teacher Training 



Teacher Training Course Design 

As already stated, the design of the course is based on two different but complementary moments: (1) face-to-face and (2) 
distance-basis, having as its rationale an interactive and problem-solving methodology and leading to a central axis relying on 
the development of interdisciplinary projects using telematic tools as enablers. 

The following Figure 2 presents, in brief, the basic structure of the course, showing how the theoretical and methodological 
studies and the technology used are articulated around the Projects in practice. 





Figure 2. Basic Structure of the Training 

As can be seen, the training is heavily constructivist-based and student-centered, aiming to develop interdisciplinary skills in 
the trainees (multipliers), so that they can use this expertise in their classrooms to train other teachers. Hence, the main target 
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of this paper is to know whether the training - albeit developed according to this constructivist paradigm - was successful in 
developing a new classroom practice, which was observed when the trainees were back to their NTEs to continue the training 
of their peers. 

Web-Based Structure of the Training 

The LEC (Cognitive Studies Laboratory) of the Federal University of Rio Grande do Sul did not use any kind of off-the-shelf 
software package in order to develop the course. As real constructivists, they preferred not be restricted to a hermetic, 
instructionist and behavioristic system, but rather to build their own Web-environment during the training. 



Since the beginning of the training, technology-mediated activities were developed via Internet services such as e-mail, 
discussion lists, newsgroups, real time communication (IRC, MOOs, CuSeeMe etc.) and spaces to store the practical and 
theoretical records of the groups, as well as contributions from the group. 



A virtual environment was created in www.psico.ufrgs.br/mec-nte2, aiming to permit exchanges among the students and 
between them and other communities. The site has a public area and a protected zone reserved only for the participants of the 
course. 



In the public space, all the navigators have the following resources at their disposal: 

• Support and Interactive resources, such as information about the course, a library offering articles and research papers, 
links to interesting sites, IRC, FAQs, a Guest Book and News about the Project; 

• Learning Projects of the Groups of Multipliers that offer partial considerations resulting from the analytical work 
addressing the developed processes and data obtained to date; 

• Individual portfolios that encompass student home pages, student diaries (relating their reflections about the student- 
teacher processes developed both in individual and collective moments); their theoretical output and a space to store 
collaborative and cooperative contributions. 

In the reserved area, one can find: 

• Group portfolios that keep texts and articles generated by discussions developed in a collective way, texts under 
production by the group, enabled by the theoretical skill that has been developed through the discussions undertaken 
in the listserv of the site; 

• Orientation for the dissertations to be developed by the NTEs’ groups. In this space, methodological suggestions of the 
tutors, reports about the implementation of the NTEs in each region and records of the exchanges between tutors and 
students are made available; 

• The Cyber Cafe — a meeting point to foment informal exchanges, jokes, invitations etc. 

On the course site, the students can upload their information through forms previously developed as templates. This 
information is immediately converted to HTML code and stored on the site. In this manner, their reflections, experiences, 
reports, critics, data to be used in shared projects and so forth can be easily deployed in the Web. 

Questionnaires 

Two types of questionnaires were used on the multipliers. The first, a quantitative questionnaire, followed the Theoretical 
Framework presented in the Research Design. The second, a qualitative one, is presented in the Annex. All of them were sent 
by Internet to the teachers and collected when the researchers were in Rio Grande do Sul State observing the multipliers 
training other teachers. All the 29 trained teachers fulfilled the quantitative questionnaire. The consolidated results led to the 
following table and exhibit: 



Table 2 

Teacher Assessment 
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ITEM 


AVG. RATING 


PHILOSOPHY 


9.92 


LEARNING THEORY 


9.92 


GOAL ORIENTATION 


9.67 


TASK ORIENTATION 


9.83 


MOTIVATION 


9.08 


TEACHER ROLE 


9.92 


METACOGNITIVE SUPPORT 


9.5 


COLLABORATION 


9.75 


CULTURAL SENSITIVITY 


9.75 


FLEXIBILITY 


9.83 




Figure 3. Teacher Assessment 

As can be seen, the socio-constructivist approach was largely understood and accepted by the teachers. The grades are very 
high though a small observation needs to be made concerning the motivation issue. The “source of motivation” ranges from 
extrinsic (i.e., outside the learning environment) to intrinsic (i.e., integral to the learning environment). Intrinsic motivation of 
instruction is elusive regardless of the delivery system, but some proponents seem convinced that WBI systems motivate 
learners automatically, simply because of the integration of music, voice, graphics, text, animation, video, and a user-friendly 
interface. Multimedia studies indicate that learners soon tire of these media elements (Reeves, 1993), as the results above 
show, and it should be obvious that motivational aspects must be consciously designed into WBI as rigorously as any other 
pedagogical dimension. 

During our visit to Rio Grande do Sul, the questionnaire presented in the Annex was also applied to the 29 teachers and the 
following results were obtained: 

Before the course: 

Most of the teachers defined their pedagogical posture as “traditional”, here understood as instructivist and teacher-centered, 
which they said was the way they were trained to be. Some of them tried to change their posture without success, as stated 
below: 



• “Before the course, I define my posture as both traditional (developed by practice) and innovative/constructivist 
(developed just in discourse). As with most of my peers my teacher-centered approach is the result of my academic 
training in college. This posture could be classified as a content-based one, taking the social reality for granted” 

• “Although I have tried to develop new pedagogical praxis, my posture was traditional, teacher-centcred and heavily 
based on knowledge transmission. I did not believe social constructivism could be applied successfully”. 

After the course: 

Most of the teachers are conscious of the need to change their pedagogical practice, however they are realistic enough to say 
that they have not yet changed. As they said: 

• “I cannot say I have changed my pedagogical posture. Though ! can say that I have reflected about my posture and am 
comfortable saying ‘I don’t know’, ‘I may be wrong’, statements which I would hardly have dared say a short while 
ago”. 

• “After the course, I believe I have not made a radical transformation, as it will take time to put this into practice. But I 
do feel myself in a process of transformation as I realize that some of my postures must be changed, and I am 
frequently aware of the need to create constructivist situations for the students. Unfortunately, most of them are rarely 
implemented”. 

Course Strengths and Weaknesses 

According to our survey the most cited Strengths and Weaknesses of the course were as follows. Strengths: The trainers’ 
sincerity and humility; the respect shown to the students and their opinions; their knowledge and their absolute coherence 
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between practice and speech. Weaknesses: The course duration which was very intensive as well as some classes given by 
“technical experts” who did not follow the constructivist approach of the course. 

Classroom Observation and Interviews 

During the visit to Rio Grande do Sul State, we had a chance to interview the multipliers and also to observe their practice in 
training other teachers. Speeches of the trained teachers conveyed to us the conclusion that they are truly aware of the need to 
change their pedagogical practice, and they see how to do that. They spoke very sincerely and the emotional component 
seemed to be very important for their forthcoming transformations. They explained that they suffered very much during the 
course, as they expected technical Cartesian training and became frustrated, as they wanted more technical skill development 
practice. Some teachers had already attended informatics courses on their own, and could not understand why these issues 
were not being addressed in the course. According to one of the teachers, they needed to be totally deconstructed to relearn 
from scratch. When asked about how they were deconstructed they pointed out two factors as the main enablers: 

• Lack of Guidelines 

Although they waited for pre-established frameworks full of landmarks, the trainers spurred them to find their own 
way in the course. This generated a considerable degree of frustration as they began to compete among themselves and 
sometimes they felt they were falling behind. These frustrations were greater among the women participants eliciting a 
profoundly emotional response. Some of them confessed that they almost quit the course in despair. 

• The Error Treatment 

Another way they were deconstructed was having to deal with a new perspective of error. The trainers said most of 
time they did not know the right answers to the teachers’ questions. This procedure confused them, and led the group 
to being obliged to live with “answers” such as: “I don’t know”, “Try yourself’ and so forth. 



We visited a training session in Gravatai, a town approximately one hour from Porto Alegre, the capital of Rio Grande do Sul 
State. There, some of the trained teachers were in the process of training some other teachers to use Informatics in their 
schools. Our observations showed that the teachers actually try to create a constructivist environment, conveying the students 
to look for the solutions themselves, in a workgroup and learning community created with the other students. At the moment 
of the visit the students were just beginning to create their own home pages on the Internet. They looked forward to having 
straight solutions and specific answers to their questions. The teachers were reluctant to give them the solutions, as they were 
more interested in getting the students to learn how to find their own solutions. This posture embarrassed some of the students, 
mainly the more Cartesian-oriented ones. Actually, it is a real battle, as the teachers have the impulse to present the solutions, 
rather than letting the students think for themselves. The teachers must be aware at all times of their former ways. 



It became apparent that the gender influence must be taken into account. According to the multipliers, the men’s reactions are 
very different from the women, as they control their emotions. This is an issue LEG wants to research deeper, so as to 
understand how it works. However what we could again see clearly is the effect of the emotional component in a socio- 
constructivist approach, even on the Web. This is an effect that cannot be disregarded and is better explained in the 
conclusions below. 

Conclusions 

It is important, before addressing any manner of conclusions, to present the assessment frame developed by LEC for its own 
course. 



LEC's Course Assessment. The qualitative assessment of the course, according to EEC’s approach, was based on the following 
elements addressing the students: 



• Regular construction and publishing of the individual and collective production on the course website; 

• Creation and publication of multimedia documents; 

• Contribution in the shared spaces on the site; 

• Publishing, follow-up and discussion of the projects developed; 

• Personal use of the website resources, both in a synchronous and asynchronous way. 



A system (AccessWatch) was used to record interaction among the students and between students and teachers. Subsequently, 
the quality of the interaction was analyzed. 



All assessment was based on the process of knowledge creation, established by analysis of student portfolios, project 
evolution, student reflections and student self-analysis developed throughout the course. 



We can conclude that according to EEC’s assessment factors, the training program was a success. The trained teachers are 
aware of the need to change their pedagogical practices, and they now know how to do that and are willing to deploy what 




they have learnt in their schools, as soon as possible. 

Theory vs. Practice 

The practical training given by the multipliers (trained teachers) to the other teachers in a cascade process complies with the 
way they were trained, i.e., they try to apply the socio-constructivist approach in the training, using Information Technology as 
a tool and enabler to give a new awareness to the teachers about how to spur the students to create their own knowledge. 

At one point one of the multipliers - almost as a throwback to her old mental model - tried to help a student by giving her the 
correct way of solving a problem. This should be considered normal, as the multipliers themselves are still undergoing their 
own transformation process which takes time. 

Care and empathy are used all the time, just in the same way as they were used in the multipliers* training. They understand 
the de-construction process the students are experiencing as they have been through the same process themselves. 

Coherence is the key-word for the success of the training, and the students see this as the multipliers* discourse and practice 
lead the course to the same central objectives, namely to use the technology to create a new modus-operandi in the classroom, 
thereby enabling the students to create and socialize their knowledge. 

Naturally, coherence is not the only issue in the transformation of pedagogical practice, as shown below, but it is paramount 
for the success of the training itself. 



The use of the framework developed by Pettigrew & Whipp (1991) depicted below in Figure 4 can help us to better understand 
the role of coherence in the success of the training and, by extension, of teacher practice. This model can show us one of the 
possible ways to achieve a successful strategic change in the educational realm. 




Figure 4. The Role of Coherence in Strategic Change 



Barriers to be overcome 



The greatest mistake one can make is to conclude that training itself is enough to change pedagogical practice in schools. 
According to the following Figure 5, developed by Morton (1996) and adapted by the researcher for the educational realm, 
several issues in the educational environment are interrelated, thus a deep and actual change affect all of them. 




Figure 5. Systemic Framework 

The diagram recognizes that organizations (in this case schools) can usefully be thought of a set of forces existing in a state of 
dynamic equilibrium. On the one hand there are its strategy (the mission it wishes to accomplish), and all the tasks that make 
up that mission. There are also its organization structure and, more importantly, the corporate culture that makes that structure 
become alive and vibrant. There are also people and the roles they are being asked to play. The fourth major set offerees are 
the information technologies. Holding all these four forces together are the management processes: the planning, the 
budgeting, and the control systems as well as the informal processes that represent the way the organization does its business. 
All these set of forces exist in an external environment which consists of the social, political, economic, and technical forces. 



Analyzing the use of ICT in the schools after teacher training in a WBI constructivist environment, we can conclude that: 



• Organization’s Strategy - if the school principal is trained in the use of ICT to develop a new pedagogical practice, it 
will become easier for the school to adopt a new praxis. However, it is important to say that the schools’ strategy must 
conform to the educational strategy developed by the state government as a whole; 

• Organization Structure and Corporate Culture - regarding school’s structure, this is perhaps the most difficult issue to 
be changed as the public authorities, themselves, have designed the school’s organizational structure. As Chandler 
(1962) said: “Structure follows Strategy”, so they are intimately linked. Hence, a radical change in the organizational 
design of the schools is essential in order to reach a new level of pedagogical practice, which is more open, 
constructivist and child-centered, otherwise new developed strategies will fail. Regarding corporate culture, the 
training gives the teachers new values, concepts and visions about education and the role ICT can play in the 
classroom. By consequence, it can lead to a (r)evolution in school practice, so as to break with the old schemata 
(mental models) and define a new vision and mission for the school. 

• Information and Communication Technology - ICT can dramatically change the pedagogical procedures and 
processes applied by the schools, by reengineering and innovating old and outdated processes; 

• Individuals and Roles - teachers trained in ICT can socialize the benefits of the new educational approach by 
explaining and demonstrating the use of ICT to their peers in the school, thus becoming catalysts in the transformation 
process. So, this is an issue the training can deal with successfully; 

• Management Processes - as the management processes depend heavily on each school principal's way of guiding 
them, once again it is perceived the importance of having the headmasters trained in how to use ICT in a constructivist 
way; 



Now, we may depict the framework already presented in Exhibit 6, highlighting the “dark” nodes of the model as the ones 
which a socio-constructivist approach associated to the use of ICT are unable to change or influence in a deeper way. So, by 
consequence, these items need to be addressed through the use of other strategies rather than via teacher training. Figure 6 
below depicts this idea and presents a holistic view of all the issues that must be changed. 
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Doj^k Node 




Figure 6. The “Dark” Nodes of the Systemic Framework 



Regarding the “Environmental Scan” (Hax & Majluf, 1991) associated to the training, it can be concluded after several 
interviews with the persons in charge of the training that political discontinuity — quite common in Brazil — can jeopardize this 
educational endeavor. Besides, as technological innovations outdated very rapidly the hardware and software acquired by the 
Government, a constant influx of investments would be necessary. Unfortunately, severe financial constraints can accelerate 
the obsolescence process of the hardware and software purchased, as well as the obsolescence process of the skills developed 
by the teachers during the training (Argote et al., 1998). 



Regarding the “Internal Scrutiny” (Hax & Majluf, 1991) associated to the training, it can be concluded that Strategy and 
Structure associated with the schools’ practice, hardly will be changed by the socio-constructivist training, as they are imposed 
in a top-down basis. These “dark nodes” will influence all the others dimensions of the model, mainly the Management 
Process. According to the teachers interviewed, a more participative approach is needed so as to allow the schools to influence 
for a better organizational structure and strategy. 



The Influence of Care in the Training Process 

We can thus see that care is fundamental in knowledge creation, as explained by von Krogh (1998). According to Maturana & 
Varela (1987), cognition is a creative act of conjuring up a microcosm. Because knowledge resides in our bodies and is closely 
tied to our senses and previous experience, we will naturally create a world in ways which are unique to ourselves. 



In this study, care engenders trust in the learning process and also gives rise to active empathy, making it possible to assess 
and understand what the other person needs. Empathy is the attempt to put yourself in another’s shoes, understanding his/her 
particular situation, interests, skill level, history of success and failures, and future opportunities and problems. By means of 
active empathy, you proactively seek to understand the other person. Through active questioning and acute observation, you 
seek out instances where your efforts are needed. You practice dialogue rather than advocating only your own point of view. 
Care accomplishes precisely the sharing of positive and negative emotions through active empathy. In this study, for example, 
the relationship between the student and the teacher in learning the use of Informatics in the classroom is made easier when the 
teacher explains that he/she has had some of the same personal frustrations in learning a specific issue, and that speed in 
grasping the difficulty of this issue is not a gauge of the intellectual capacity of the student, but one of the inherent 
characteristics of a long learning process. 

Hence, it can be concluded that the approach developed in this specific training for knowledge creation was based on von 
Krogh, (1998): 
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Characteristics of Care in the Case Study Model 



PROCESS 


LEVEL 


CHARACTERISTICS 


BESTOWING 


INDIVIDUAL 


Knoiivledge created in a supportive eiT/ironment ("maximum leverage”) 

Strona intent to share knovviedae on the future 

Feedback from others 

Integration of individuals into the team 


INDWELLING 


SOCIAL 


"Equals" creating knowledge 

Questioninq and chanainq the basis for leoimate knov'/ledae 
Sharing to help the team grow 

Attempts to "look with" not "look at" other team members 



As there is care in training, there is mutual trust, active empathy, access to help among team members, lenient judgments 
towards participants in the team, and courage. In such a situation the student will bestow knowledge on others as well as 
receive active help from others (others bestowing knowledge on him/her). The environment is supportive, and the goal of the 
learning process shifts from obtaining “maximum grip” to reaching “maximum leverage” on others’ knowledge. There is a 
mutual intent to help others to optimize their task performance, and, therefore, to share knowledge. The individual can 
experiment more freely in order to develop unconventional task solutions, and is emboldened in the pursuit of knowledge 
creation. 



The individual is integrated into the team. Other students and teachers take an active interest in the learning process, and the 
individual is encouraged to make knowledge explicit while learning. When care runs high, colleagues show interest and 
support, and the individual member can spontaneously articulate his/her knowledge using unconventional language, 
metaphors, and analogies. Expressing personal difficulties in knowledge creation will be met with lenience from other team 
members and active feedback will be provided. 

The process of mutual bestowing provides fertile ground for a distinct process of creating social knowledge in a team: 
Indwelling. Indwelling is of particular importance to the sharing of tacit knowledge and concept creation. Polanyi & Prosch 
(1975) suggested that dwelling in a concept can be understood as a dramatic shift of perspectives, changing the concept from 
“looking at” to “looking with”. In broader terms, indwelling is about commitment to an idea, to an experience, to a concept, or 
to a fellow human being. In developing shared tacit knowledge, the challenge for the teachers in knowledge creation is to 
dwell in the experiences, perspectives and concepts of other participants, or in other words, to change from self-commitment to 
commitment to others. In changing such deeper-level commitments, participants literally make changes in their perceptions. 
When care runs high, team members extend help to each other in finding new means of conveying and sharing personal 
beliefs. Trust and lenience make it easier to articulate emotional aspects of an experience, as was realized during this research. 

Again, it is important to stress that the emotional component, here translated as care, was a vital force for the success of this 
socio-constructivist WBI training. Socio-constructivist training demands by its very nature high involvement of the teachers, 
both in the face-to-face stage and the distance-basis one. As the training aims to lead the students to reprogramming their 
minds by challenging their “temporary certainties”, some physical as well as emotional support is necessary. The students are 
conveyed to an “un-leam” and “re-leam” process, and sometimes they offload their mental models without having acquired a 
new schemata. At this moment care is absolutely necessary (von Krogh, 1998) as they are emotionally deconstructed. 



Traditional vs. Socio-Constructivist Web-Based Training 

As a summary of our conclusions the following Table 4 is presented comparing some issues addressed in traditional teacher 
training and in a socio-constructivist program: 



Table 4 

Traditional vs, Socio-Constructivist Approach 
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TRADITIONAL TRAINING 


ANALYZED lylODEL 




Knowledge Transmission 


Knowledge Creation 


General Characteristics 


Hierarchical 


Team-based 




Dire ctive 


Relational 




Fixed Roles 


Flexible roles 








Objectives 


Defined by the T eacher 


Negotiated among the players 










Fixed 


Flexible and changeable 


Contents 


Mono-disciplinary 


Interdisciplinary 




Based on texts, books etc. 


Based on Internet, the peers etc. 








Methodology 


Based on reproduction 


Interactive and problem-based 




Cartesian and Taylorist 


integrated 








Evaluation 


Based on command-control 


Meta cognitive-b ased 



Final Recommendations 

As final recommendations that can improve the training analyzed above, we should highlight the following issues to be 
implemented: 



• Mentoring Programs 

Mentoring programs must motivate the trained teachers to share their knowledge with junior teachers and newcomers 
in the schools. Management can achieve this by defining two sets of responsibilities for each individual, namely to 
develop proportionally his/her ability to acquire expertise, in line with the responsibility to make his/her help 
accessible to those who need it as his/her expertise grows. 

• Headmaster Training 

As stated above, the role of the headmaster in the creation of a “new school” is paramount, not Just to motivate the 
teachers, but also to be fully involved in the other realms that hinder a true, radical transformation in the pedagogical 
practice, as presented in Figure 6. Hence, their participation and involvement in the training process is highly 
important for training targets to be fully accomplished. 



• Social Events 



It is important to reunite the trained teachers in social events, where in a face-to-face basis they can share the 
experiences, improvements and obstacles they are facing. These moments are as important as opening and maintaining 
a newsgroup on the Internet the participants of which are already trained in this socio-constructivist approach. 



All in all, this was a very innovative teacher training initiative which fully complied with the trainers’ paradigms and 
pedagogical philosophies, seeking coherence at all times. Undoubtedly, as stated before, this alone will not change the 
outdated former teaching and learning praxis, but it is an updated and powerful enabler for instilling the necessary changes in 
the other remaining areas addressed in the whole educational process. 
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Abstract 

Districts play a critical role in reforming schools. In January 2000, NYC community school 
districts applied for Title I, I ASA, funding to carry out corrective actions against historically low- 
performing schools. Our purpose was to examine (a) how districts planned to take corrective 
action to address problems that cause low performance; and (b) the extent to which school choice 
could be implemented in those districts which were applying for corrective action funding. 

Districts most commonly identified teacher turnover, poor-quality instruction, and student needs 
as causes of low performance. In response, districts proposed providing professional 
development related to instructional strategies, but often ignored other important issues. 

Moreover, most districts described plans to take corrective actions that would decrease schools’ 
decision-making authority, but then failed to identify steps to increase the districts’ own capacity 
to execute greater responsibility once control had been taken from the schools. Districts overall 
seemed unable to implement school choice plans in an effective manner. 

As part of the FY 2000 funding for the Elementary and Secondary Schools Act of 1 965 (ESEA), Congress 
appropriated $134 million for local educational agencies to bolster their capability for carrying out school 
improvement and corrective action responsibilities under section 1 1 16 (c) of Title I, lASA, and to provide students 
enrolled in low-performing schools the opportunity to transfer within the district to another school not identified as 
low-performing. This appropriation was aimed at strengthening accountability for student performance results, and 
was seen as “a key reform strategy capable of delivering a high-quality education for all students” (U. S. Department 
of Education, 2000, p. 2). Given the recent passage of the reauthorization of the ESEA in late 2001 , including its 
provisions for districts to take corrective actions against low-performing schools, it seems appropriate to examine the 
manner in which some districts in New York City (NYC) have proposed to leverage change in historically low- 
performing schools. 
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One clear intention of the 2000 appropriation was to have districts play an important role in turning around their low- 
performing schools. Districts were obligated to help low-performing schools develop and implement school 
improvement plans, and to provide additional professional development and technical assistance. Districts were also 
authorized to take one or more corrective actions which could include withholding funds from identified schools, 
decreasing decision-making authority of school leaders in identified schools, or reconstituting school staff (see 
Appendix A for the complete list of corrective actions). An important provision of this appropriation was that, when 
necessary, district leaders could use improvement funding to build up their own capability for turning around low- 
performing schools. 

In addition to improvement and corrective action responsibilities, districts accepting these funds in FY 2000 were also 
required to implement school choice plans. School choice was intended to offer students in a chronically low- 
performing school the opportunity to attend a better school the following year. If districts were unable to offer school 
choice to every student desiring this option, provisions had to be made to transfer as many students as possible, and to 
allocate existing opportunities on an equitable basis. School choice was viewed as a means to provide students with 
better educational opportunities and to increase involvement of parents in children’s education (U. S. Department of 
Education, 2000). 

Strategies for Turning Around Low-Performing Schools 

Corrective action and school choice are both federal strategies intended to turn around low-performing schools. 
Corrective action provides districts with the authority to intervene directly in schools. With this legislation, districts 
were under special obligation to take corrective action against schools that failed to make adequate yearly progress 
toward student achievement goals in English Language Arts (ELA) and Mathematics for two out of three years after 
being identified for improvement. Section 1 1 16 (c) of Title I, I AS A, gives districts a pivotal role in helping turn 
around low-performing schools. To do so, however, districts must have personnel with the knowledge and skill to 
help schools adopt and implement effective strategies for improvement. 

Likewise, school choice is an accountability strategy intended to bring market forces to bear on low-performing 
schools. The public notification of poor performance, and the dissatisfaction accompanying parents’ transfer of their 
children to higher performing schools creates public demand for better education. Like corrective action, districts 
must have a level of capacity for this strategy to be effective. In this case, districts must have space in schools that are 
making adequate progress in order to allow the exercise of the choice option. 

District leadership for building school capacity and improving student learning. The path to improving chronically 
low-performing schools represents virtually uncharted territory for many school leaders. District leaders appear to 
lack the expertise and resources to respond to school-level calls for assistance, according to a recent report on district 
school improvement efforts (U. S. Department of Education, 2001). Given this, schools against which districts are 
planning corrective actions may be in a particularly troubling situation where they have been unable to improve over 
an extended period of time, and their district is ill-equipped to help halt their decline or provide the necessary external 
support to assist and guide improvement. 

It is unclear exactly how districts will carry out corrective actions against schools, or the effect such actions are likely 
to have on student performance. Only a limited amount of empirical evidence exists to guide district-led reforms. 

What seems clearer, though, is that if districts have not been able to assist schools in need of improvement, there is 
little chance they will be able to turn around low-performing schools by taking corrective actions unless some action 
is taken to boost the capacity for reform at the district level. 

One strategy districts might use to help low-performing schools is to use professional development funding in ways 
that directly affect teaching and learning. Youngs (2001) suggests that districts can use professional development to 
enhance the capacity of schools to improve student achievement. A school’s capacity is enhanced when professional 
development includes features that improve teacher knowledge and skill, builds professional community within a 
school, and fosters coherence of instructional programs while providing some autonomy. These features provide 
knowledge and support for improvement, and ultimately make it more likely that teachers will change their 
instructional practice in ways that will improve students’ academic achievement. Applying Youngs’ criteria for 
effective professional development provides a guide with which districts can judge professional development, and 
provides a means to tie professional development directly to student learning. 

A related strategy that may hold some promise for helping low-performing schools is for superintendents to become 
instructional leaders. Petersen (1999) described the actions of eight California superintendents who are credited with 
turning around many low-performing schools in their district. These superintendents all embraced the role of 
instructional leader at the district level, and reorganized district priorities and structure to focus on teaching and 
curriculum in schools. Petersen described superintendents who articulated a vision of good teaching for their districts, 
held principals accountable for carrying this vision into the classrooms of their schools, and evaluated principals on 
their ability to act as instructional leaders. 

These district-level strategies are consistent with what Elmore (2000) described as distributed leadership. According 
to Elmore, school leaders can improve low-performing schools by tightly coupling policy, administration, and 




teaching with standards for student learning and performance. This tighter coupling allows leadership for student 
learning to be distributed across the multiple levels and roles in districts, schools, and classrooms. 

According to Elmore (2000), the task of improving student learning needs to be shared, or distributed, among every 
person in education organizations, and individuals must contribute their expertise and be accountable in a manner 
appropriate to their level. For example, superintendents are accountable for system organization, allocation strategies 
focused on instruction, and principal evaluation. Principals are accountable for school improvement strategies, 
professional development, and teacher evaluation. The specific role for superintendents, supported by policy, would 
be to arrange conditions and assure training and support for principals that would allow them to be instructional 
leaders in their school. With this support, principals, in turn, would arrange conditions, and provide training that 
would allow teachers to provide high-quality instruction; and teachers would provide conditions and training to 
support student learning. 



District leaders focused on the core technology of education (instruction and curriculum) and a tighter collaboration 
between superintendent and principals are two themes that appear to be gaining support in the most current literature 
on school leadership (e.g.. Hatch, 2001; Institute for Educational Leadership, 2001 ; Spillane, Halverson, & Diamond, 
2001). These strategies seem to be a promising area in which districts might build their own capacity for turning 
around low-performing schools. 



Parents ' role in improving low-performing schools. School choice is a reform strategy that, over the years, has taken 
many forms (e.g., charter schools, magnet schools, vouchers for private schools, inter-district school choice, and intra- 
district school choice). In all its manifestations, the strategy is intended to directly involve parents in holding schools 
accountable for improving. By providing parents with a choice about the schools their child attends, advocates believe 
competition and market forces will force low-performing schools to improve. To date, however, there is little 
conclusive evidence to support or deny this claim (e.g., Goldhaber, 1999; Myers, et al., 2000). 



Almost 10 years ago. New York City parents were given the option to transfer their children to any other school in 
any other district within the City (i.e., inter-district transfer option). Parents could exercise choice provided space was 
available in the receiving school and transportation could be arranged. Relatively few students, however, actually had 
the opportunity to attend schools outside of their home districts due to high demand for schools with good reputations, 
and because of the daunting procedures for obtaining permission to transfer to another district (Teske, et al., 2000). 
Teske et al. believed that because students were rarely able to exercise the choice option, inter-district choice in New 
York City has had a very limited effect on low-performing schools. 

In another New York City choice experiment, vouchers to attend private schools were given to a random sample of 
1,300 children of low-income families. Meyers et al (2000) reported that parents believed the environments for their 
children’s education had improved (i.e., less racial conflict, less fighting, more homework). In the two-year report of 
progress, however, there appeared to be few differences in performance on standardized tests between voucher and 
control students. 

On the positive side, Teske et al’s (2000) analysis of two New York City districts’ intra-district choice plan provides 
compelling evidence that school choice within a district may provide parents and students with meaningful education 
alternatives, and also help improve low-performing schools (Teske, et al., 2000). Teske et al traced the reading and 
math achievement of students in two New York City districts (Districts 4 and District 2) that adopted school choice 
plans decades earlier than required by federal law. The authors demonstrate that the introduction and increase in the 
percentage of available schools for choice was positively related to gains in student performance in the district. 



School choice, as a strategy for improving low-performing schools, continues to receive popular and critical support 
(e.g.. Hart & Teeter, 2001). The effectiveness of school choice options seems to hinge, however, on the extent to 
which parents and students can actually exercise choice. In cities or districts where only a few schools have been 
identified for improvement or where enrollment is below 1 00% capacity, school choice may be a powerful reform 
strategy. In districts where few choice options exist, the school choice strategy may be ineffective. 

Purpose of the Analysis 

The purpose of this analysis was to describe the actions proposed by NYC districts to improve schools against which 
corrective actions must be taken, and to describe the extent to which districts would be able to honor school choice 
requests. This information may be helpful in evaluating current strategies for turning around low-performing schools, 
and in helping to direct other districts’ action in the future as more and more schools are identified for corrective 
action. 

Methods and Procedure 



ERIC 



This report is a descriptive analysis of written applications submitted by New York City Community School Districts 
to the New York State Education Department (NYSED) in January, 2000. Districts were applying for a portion of 
$12.8 million that was set aside for districts in New York State. In New York City, these funds were designated to 
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support district efforts to improve low-performing schools and to take corrective action. 

Twenty-five districts in New York City applied to NYSED for funding to carryout corrective actions against 122 
schools (Elementary = 64; Middle = 54; K-8 = 4) that failed to make adequate yearly progress toward student 
performance goals on the NY state assessment in English Language Arts (ELA) (52%), Math (9%), or both EL A and 
Math (39%). To apply for funding, districts attached an amendment to their yearly District Comprehensive Education 
Plan. The content of the amendment included a list of schools against which corrective actions were to be taken; an 
analysis of the needs and priorities in the schools and the district; a description of the district level intervention for 
assisting identified schools and for building district capacity for assisting schools; and a detailed description of the 
districts school choice plan (see Appendix B for questions on the applications). 

Categories for analyzing district applications were developed directly from district responses, and from constructs in 
current leadership literature. These categories were used to summarize districts’ analysis of schools’ needs, and their 
proposed actions for improving achievement in English language arts (ELA) and Mathematics. We also examined 
how districts operationalized the corrective actions they proposed to take, and examined the school choice plan in 
terms of the likely number of students who could exercise their school-choice options. 

Results 

The results from our analysis will be presented in sections that correspond to the sequential order of questions found 
in the district application (see Appendix B): school needs identified by the district, proposed district action, and 
proposed corrective action. Finally, results from an analysis of districts’ school choice plan will be presented. 

Needs that Districts Identified 

For descriptive purposes, we grouped districts’ identification of school need into five broad categories, and then 
identified specific needs within each category (see Table 1). The school need most commonly identified by districts 
was the need for teacher stability in low-performing schools. The need for teacher stability refers to a high rate of 
teachers transferring in and out of the low-performing schools. Eighteen out of twenty-five districts (72%) identified 
this as a need that impedes progress toward improving student performance. The second most frequently identified 
areas were student need (68%) and improved instructional strategies (68%). Student need represents challenges to the 
instructional organization of schools presented by students who, for example, have special learning needs, are English 
language learners, or who frequently transfer between schools, etc. The district-identified need to improve 
instructional strategies was primarily described as a need fro teachers to provide differentiated instruction that would 
accommodate student differences. This need, as described by the districts, typically did not address more stable 
student factors, such as poverty or mobility. Curriculum alignment (64%), parent involvement (60%), qualified and 
certified teachers (56%) and principal leadership (56%) all were identified as school needs by a majority of districts. 
Although these needs do undoubtedly create barriers to school improvement, districts rarely identified other needs 
that also may have a negative impact on students’ achievement. 

Table 1 



School Needs Identified by the Districts 


School needs 


Number of districts identifying the 
need 


Percentage of districts identifying 
the need 


School-level needs 




Student needs 


17 


68% 


Parent involvement 


15 


60% 


Leadership needs 




Principal leadership 


14 


56% 


District leadership 


10 


40% 


Instructional needs 




Instructional strategies 


17 


68% 


Curriculum alignment 


16 


64% 


Training needs 





W2 



Access to professional 
development 


11 


44% 


Personnel needs 




Teacher stability 


18 


72% 


Qualified teachers 


14 


56% 


Principal stability 


10 


40% 



For example, fewer than half of the districts identified principal stability (40%) as a need in schools requiring 
corrective action. Moreover, as few as one-quarter of districts identified needs related to high-quality professional 
development (24%). Even fewer districts said there was a need for teachers to be able to collaborate with one another 
(8%); increased teacher leadership (4%); improved social services for students (4%); or the need to develop a reliable, 
qualified pool of substitute teachers (4%). Many of these needs correspond to factors that help create a positive, 
supportive school climate, an that contribute to a school culture focused on teaching and learning. It is unclear from 
the analysis of these applications whether these needs did not exist in the designated schools, or whether the majority 
of application authors were simply not aware of these needs. 



Proposed District Action in Response to the Needs Schools 

Districts’ responses to the most common school needs were also grouped into five overarching categories and then 
specific examples identified in each category (see Table 2). Overall, the grouping of these proposed actions revealed 
an interesting pattern of responses from the districts that seemed to ignore many of the most significant problems they 
had previously identified (e.g., teacher stability), and also any opportunities for increasing the capacity of district 
personnel. 



District improvement action. One purpose to which corrective action funding could be put was to build the capacity of 
local districts to take action that would help turn around low-performing schools. Districts most often proposed two 
types of district-level improvement actions (see Table 2). 

Table 2 



Actions and Tally of Districts Proposing Responses to School Needs 


Proposed action 


Number of districts 
proposing 


Percentage of districts 
proposing 


District improvement action 




Resource decisions 


18 


72% 


Monitoring 


9 


36% 


Instructional strategies 


7 


28% 


Use of data 


5 


20% 


Curriculum alignment 


4 


16% 


Team leadership 


4 


16% 


instructional improvement actions 




Implement/revise academic program 


15 


60% 


Implement/revise instructional time 


10 


40% 


Teacher improvement actions 




Instructional strategies 


21 


84% 


Curriculum alignment 


13 . 


52% 
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Principal improvement 




Instructional strategies 


6 


24% 


Team leadership 


6 


24% 


School organizational improvement 




Enhancing parent/community 
involvement 


15 


60% 


Assist with school planning 


12 


48% 



First, districts proposed taking the important step of reallocating resources (72%). Typically, reallocation of resources 
meant reassigning existing staff-development personnel, or hiring consultants to do training. In some districts, 
resource allocation meant hiring specific individuals for the district to address specific needs, such as data analysis, 
principal training, or teacher professional development. One district, for example, proposed to improve teacher 
recruitment by hiring a former principal to act as a liaison between the district and area universities with teacher- 
training programs. Second, districts proposed increasing the amount of time and frequency of district monitoring in 
corrective action schools (36%). Monitoring was described, for example, as having the superintendent stop into 
schools more often or district-level staff attending staff meetings in the schools. 



Some districts did propose actions that would increase the capability of district personnel to respond to the needs of 
low-performing schools, but these plans were present in less than a third of the applications. For example, a few 
districts proposed obtaining professional development for staff developers and curriculum directors in instructional 
strategies (28%), and the use of data (20%). Fewer proposed training district personnel in curriculum alignment (16%) 
team leadership (16%), instructional planning (0%) or assessment alternatives (0%). These actions, in particular, seem 
to be critical for district personnel if they are to lead school efforts to improve student achievement. 



Instructional programs/practices improvement. Districts commonly proposed two actions aimed at improving the 
instructional programs in schools (see Table 2). Districts most often proposed implementing new academic programs 
(e.g., Reading Recovery, Saturday Math Academy) and revising existing programs (60%). Fewer districts proposed 
revising curriculum in ELA and Math (24%), or implementing a test-taking curriculum (28%). Forty percent of the 
districts proposed reallocating the amount of time spent on particular areas of instruction (e.g., 90 min literacy 
blocks). None of the districts proposed providing common time for teachers to plan or revise instruction as a strategy 
for improving instruction. 



Teacher improvement. Districts’ teacher-improvement actions were centered on providing professional development 
in several important areas (see Table 2). The two most commonly proposed actions were providing professional 
development for use of specific instructional strategies (e.g., balanced literacy instruction, constructivist math 
instruction) (84%), and alignment of curriculum to state standards (52%). Relatively few districts proposed to provide 
professional development aimed at helping teachers use assessment data (28%), or to boost team leadership (1 2%). 
Mentoring (12%) and school/district supports for obtaining certification (8%) are two strategies often used to improve 
teacher quality and stability in a school, but these two approaches were proposed by only a few districts. This is 
especially striking given the number of districts identifying teacher stability as a need in low-performing schools. 

Principal improvement. Taking actions intended to improve principal leadership appeared in about one-quarter of all 
the district applications (see Table 2). Those districts that did include it most frequently recommended training 
principals in instructional strategies (24%) and team leadership (24%). Training principals to use data to inform 
decisions (16%) and to align curriculum (16%) was mentioned less frequently. These results are somewhat surprising 
given the heavy emphasis on improving teachers’ instructional practice, and it may indicate that districts are 
overlooking important strategies for turning around their low-performing schools. 

Student need-based improvement. As few as 24% of districts proposed implementing or revising some type of 
program to respond to student needs. The actions districts proposed included establishing/strengthening ties to 
community-based organizations in order to provide after-school recreational opportunities for students; providing 
teachers with professional development aimed at reducing suspensions; and requiring the district director of student 
support services to meet with school guidance counselors. Although district responsiveness to student need appears 
modest, they were often addressed with additional academic programs, such as extended day, and linking remediation 
services to the regular education program (see Table 2). Still, the modest number of districts planning to address 
student-need issues is striking given that so many districts (68%) identified specific student characteristics as barriers 
to improving student performance. This figure may indicate that district leaders felt they were unable to respond 
directly to these needs. 
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School organization improvement. Finally, districts’ plans for improving low-performing schools tended to focus 
primarily on teacher and instructional improvement (see Table 2). At the school-level, however, districts did propose 
taking some steps to improve parent and comn^^ involvement in schools (60%), and several districts 



recommended helping schools prepare improvement plans (48%). Districts did not frequently propose larger 
structural changes, such as reorganizing grade configurations (0%), changing from an age-based to an ability-based 
grouping (28%), or creating smaller class-sizes (16%). 

Proposed Corrective Actions to be Taken Against Low-Performing Schools 

The Improving America’s Schools Act (1994) suggests eight corrective actions that may be taken by districts against 
schools that have failed to make adequate yearly progress for two out of three years following their initial 
identification for improvement. In the current applications, districts on average proposed taking slightly less than 2 
actions against low-performing schools (Af = 1.76, SD = 1 .27). Four districts (16%) did not specify any corrective 
action to be taken. This may indicate that districts leaders knew of a variety of possible improvement strategies, but 
were unclear how these strategies “mapped onto” the corrective action options. This limited response may also 
indicate that district leaders did not perceive “corrective action” to be distinct from previous school improvement 
strategies. 



District descriptions of their corrective actions. The corrective action that districts most frequently proposed was to 
decrease the decision-making authority of the school leaders (see Table 3). Districts operationally defined “decreased 
decision-making authority” in a variety of ways, but most tended to revolve around districts requiring the adoption of 
instructional schedules (e.g., literacy block), the adoption of a academic programs (e.g., Saturday Math) and 
curriculum, or mandating teacher participation in professional development activities chosen by the district. Districts 
also decreased decision-making authority by specifying how budgets would be structured, and by increasing oversight 
of a principal’s decisions related to literacy and math instruction, or oversight of comprehensive improvement 
planning. It seemed unclear, however, whether these corrective actions represented a unique approach to improving 
low-performing schools. 



Table 3 



Corrective Actions Proposed in District Applications 


Proposed corrective action 


Number of districts 
proposing action 


Percentage of districts 
proposing action 


Decreasing decision-making authority 


18 


72 


Authorizing student transfer 


6 


24 


Reconstituting school staff 


5 


20 


Creating interagency agreements 


5 


20 


Withholding funds 


4 


16 


Revoking school wide program authority 


2 


8 


Making alternative governance arrangements 


2 


8 


Implementing opportunity-to-leam standards 
and strategies 


1 


4 



To a lesser extent, districts also proposed authorizing students to transfer out of low-performing schools, 
reconstituting school staff, and creating interagency agreements. The number of districts intending to allow students 
to transfer as a corrective action (24%) is approximately equal to the number of districts that created transfer policies 
in response to this funding opportunity (see next section on School Choice). This may indicate that these districts 
were beginning to implement School Choice in response to the current legislation. 



The action of reconstituting staff typically included monitoring personnel in low-performing schools, and hiring new 
principals and teachers — as one district application stated — “if possible and necessary.” Under the reconstitution 
action, districts proposed making changes to personnel roles (i.e., changing an administrative position to a teaching 
position), and proposed hiring new staff (e.g., an assistant principal with expertise in literacy, a new librarian). 
Creating interagency agreements was not typically about creating new agreements, but rather about reviewing, 
improving, or strengthening existing collaborations. 

The four districts that proposed withholding funds (16%) tended to operationalize their actions in terms very similar 
to those used to described decreased decision-making (e.g., withholding funds to meet district professional 
development goals; taking over the budget-making process if schools were found to be fiscally irresponsible). The one 
district that proposed implementing opportunity-to-leam standards or strategies did not describe what proposed 
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strategies corresponded to this corrective action, or other ways they intended to carry out the action against the school. 

Overall, corrective actions proposed by districts did not seem to address the pressing issues in the schools. That is, 
corrective actions were primarily concerned with decision-making around professional development, program 
selection, and budgets despite the fact that districts most frequently expressed the need for greater stability in the 
teaching staff- a need that directly impacts the effectiveness of professional development, and indirectly affects the 
quality of instruction. Similarly, professional development undoubtedly would be concerned with specific content 
areas, but it is noteworthy that only one district (4%) identified “opportunity-to-leam standards** as an action to be 
taken against a school - an action that explicitly addresses student achievement. 



School Choice Plans 



The final portion of the state application required districts to describe how they would implement a school choice plan 
for students in low-performing schools. Fifteen districts (60%) stated that they already had pre-existing choice plans 
that would satisfy the requirement specified in the law. Five districts (20%) articulated plans that were in response to 
the legislation guidelines, and five districts (20%) reported that there was no plan for school choice, or that conditions 
existed which made it impossible to create and implement a school choice plan. Conditions that prevented districts 
from implementing a choice plan included potential receiving schools that were already overcrowded, and a lack of 
schools within the district that were not already identified as in need of improvement. 



Sixteen districts (64%) said they could implement a school choice plan. Within those districts, there are an average of 
4.88 (SD = 2.43) schools into which students could transfer. No district, however, specified exactly how many 
students could transfer into each school under the school choice plan. Nine districts (36%) reported that they could not 
transfer any students. Five districts (20%) had the capability to allow students to choose to attend one receiver school 
- hardly enough to accommodate all students who might wish to exercise school choice in even one low-performing 
school. Ten districts (40%) seemed to have the capability of allowing students to choose to attend between two and 
ten schools, but again the number of choice students each school could accommodate was not specified. Only one 
district, which had a small number of low-performing schools, appeared to have the capacity to allow all students in a 
low-performing school to exercise a choice option. 

Discussion 

The special appropriation of $134 million in FY 2000 was intended to strengthen school and district accountability for 
student performance results. One hundred percent of the funding was directed to local education agencies for the 
purpose of school improvement, including taking corrective action against historically low-performing schools and 
implementing intra-district school choice programs. An important provision of this appropriation was that these funds 
could be used to enhance the capacity of local education agencies to carry out its obligations to improve low- 
performing schools (USED, 2000). 

This analysis is particularly timely given the current reauthorization of ESEA. The reauthorized bill continues to 
require districts to take corrective action, and to provide for intra-district choice once a school has been identified for 
improvement, but it also places even greater responsibility on districts for improving the lowest performing schools. 
The greater expectations for effective district action are seen in at least two ways. First, the reauthorized bill reduces 
the amount of time that elapses between a district identifying a school for improvement and taking corrective action. 
Second, one year after a district has taken corrective action, the school must make its adequate yearly progress goal, 
or be subject to an alternative governance agreement (e.g., reopening as a charter school, replacing school staff, 
contract with a private management company). Both of these represent changes that require district leaders to 
intervene sooner and to act with greater effectiveness than ever before. This descriptive analysis of NYC districts’ 
proposed corrective actions offers some indication of the challenges and pitfalls school leaders will likely face as they 
attempt to improve their schools. 

School Need and District Response 



The needs most often identified by districts were the lack of teacher stability or staffing difficulties in the low- 
performing schools, teachers* use of ineffective instructional strategies, and student need (e.g., poverty, mobility, 
limited literacy experiences, limited English proficiency, and special education). In response to school needs, districts 
most often proposed more professional development for teachers in instruction and curriculum, and proposed to 
reallocate existing district resources to provide professional development and monitor schools. Although these 
strategies may be effective, several other important improvement strategies were ignored by a large proportion of 
districts, including improving principal leadership and improving district capacity to assist low-performing schools. 

In addition, there appears to be a significant misalignment among school needs, district actions, and effective reform 
strategies. For example, instructional leadership of principals may be an essential component of improving low- 
performing schools (e.g., Berends, et al., 2001; Petersen, 1999; Institute for Educational Leadership, 2001; Elmore, 
2000), yet principal improvements were proposed in only one-quarter of the districts’ applications. 
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It is also disconcerting that districts perceive the causes of low-performance (e.g., poor instructional strategies) and 
the strategies for addressing it (e.g., professional development for teachers) to lay squarely within the control of 
teachers, while at the same time, teacher leadership and time for teachers to collaborate is not perceived to be a need 
in many districts. It is ironic that districts expect teachers to improve their instructional practice, receive training and 
adopt new practices, but then provide no new instructional leadership at the district and building level, or time for 
teacher collaboration. Given the high rate of teacher turnover reported by the districts and by the entire city 
(Brumberg, 2000), it seems doubtful that expenditures for more teacher professional development and for consultants 
will have the desired effect on instruction or stem the flow of personnel. 



More important from a district perspective, the corrective actions that were proposed also seemed to ignore some 
important consequences that would follow such actions. The corrective action districts most often proposed taking 
against low-performing schools was to decrease the decision-making authority of the school leader. Decreasing 
decision-making at the school level may be an appropriate response given the poor performance of many of the 
schools. Yet districts do not seem to be aware of the need to improve their own capability. The applications indicated 
that districts were proposing to take over greater decision-making related to the implementation of instructional 
programs, adoption of curriculum, teachers’ professional development, and to increase oversight of principals. These 
actions, although potentially effective, seem to be less likely to succeed due to the fact that districts were taking on 
greater responsibility for school- level orchestration while at the same time not addressing the capacity-needs of 
district personnel. 



If districts are to have a chance at turning around the corrective action schools, it is important for district-level 
personnel to be well-trained in the important areas of instruction, instructional leadership, assessment and use of data, 
parent involvement, and communication. Apparently, district leaders felt confident in their ability to shoulder these 
responsibilities. A recent report on the district-role in school improvement, however, suggests otherwise (U. S. 
Education Department, 2001). 

School Choice Plans 

The effectiveness of the school choice strategy for improving low-performing schools rests upon the ability of parents 
and students to exercise their option to escape low-performing schools. Based upon the summary of district 
applications, it seems this strategy is likely to be ineffective in many of the applying districts. Eighty percent of the 
districts applying for the grant had or were implementing an intra-district choice plan. Unfortunately, factors such as 
overcrowding and too many low-performing schools left far fewer districts actually able to implement the choice plan. 
Of those implementing, only one district reported having the capability to allow large numbers of students to choose 
another, better-performing school. As was the case with inter-district choice in New York City and Milwaukee, this 
strategy seems unlikely to bring competition to bear on low-performing schools because very few parents and 
students could actually exercise choice (Teske, et al., 2000). 

Conclusion 

Overall, the patterns identified by this analysis suggest that districts are on the right track for improving low- 
performing schools by focusing on instructional quality. This analysis also suggests that this improvement strategy is 
likely to be ineffective in many districts where superintendents do not arrange conditions to focus on the core 
technology of schools (e.g., distributing learning leadership, principal training, teacher time to learn new strategies). 

This analysis also suggests that intra-district transfer choice has little chance to improve schools in districts with a 
high incidence of low-performing schools. The requirement to exclude low-performing schools from choice options 
could be lifted to increase the likelihood that competitive forces be brought to bear on schools as was the case in 
District 4 and District 2. It is unclear, though, what effect this move would have on improving schools absent any 
other efforts that focus on instruction and learning. 

Finally, it seems there is relatively little that is unique to the corrective actions described in these applications. Many 
of the actions districts proposed (e.g., professional development, program initiatives) were very similar to those 
already enacted by districts attempting to improve low-performing schools. The one feature that seems to distinguish 
corrective action from others is the control, authority, and responsibility placed upon the district. The districts making 
these applications, however, seemed to make few plans that reftected this new, more intense leadership role. 
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Appendix A 



Section 1116 of Title 1 in the Improving America’s Schools Act (1994) 

Sec. 1116. ASSESSMENT AND LOCAL EDUCATIONAL AGENCY AND SCHOOL IMPROVEMENT 

• CORRECTIVE ACTION. - (A) . . . local education agency may take corrective action at any time against a 
school that has been identified under paragraph (1) but, during the third year following identification under 
paragraph (1), shall take such action against any school that still fails to make adequate progress. 

• (i) Corrective actions are those, consistent with State & local law, determined and made public and 
disseminated by the local education agency, which may include - 

• withholding funds; 

• interagency collaborative agreements between the school and other public agencies to provide health, 
counseling, and other social services needed to remove barriers to learning; 

• revoking authority for a school to operate a schoolwide program; ^ 

• decreasing decision-making authority at the school level; 

• making alternative governance arrangements such as the creation of a public charter school; 

• reconstituting the school staff 

• authorizing students to transfer, including transportation costs, to other public schools served by the local 
educational agency; and 

• implementing opportunity-to-leam standards or strategies developed by such State under the Goals 2000: 
Educate America Act. 

Appendix B 

Items Analyzed from New York City Application for 2000-2001 Title 1 Improvement and Choice Funds 

Needs Identification 

■ Research has shown that there are often organizational or systemic factors that negatively impact progress in 
low performing schools. Describe the factors affecting student achievement in Title 1 Corrective Action 
Schools. Such factors might include district policies and procedures, budgets and resource allocation, 
technical assistance, etc. 

■ What were the results of the needs assessment? Describe the priority areas that emerged from the needs 
assessment that need to be addressed through the application amendment. 

District Action 

■ Explain how the district-level organization, structure, and comprehensive plans will support a focused district 
intervention to assist identified schools in improving achievement in English language arts and/or 
mathematics. Where such support is not already in place, describe how district capacity will be built and 
district level changes made under this grant to better provide support to Title 1 Corrective Action Schools. 

■ Describe the corrective action steps the district will take for identified Title 1 Corrective Action Schools as 
required in lASA, Section 1116 (c). 

Public School Choice 



■ List the schools not identified for SURR, Title 1 Corrective Action, and Title 1 School Improvement. 

■ Does the district have an existing policy allowing for public school choice? 

■ 

o If yes, please attach and explain how the policy will be used to meet the school choice provision of 
this amendment. Include a timeline for implementation under this amendment, 
o If the district does not have an existing transfer/choice policy, describe how it will develop and 
implement a program of public school choice. Include how the district will provide all students in 
schools identified for SURR, Title 1 Corrective Action, and Title 1 School Improvement, with an 
option to transfer to a public school within the local education agency, including public charter 
schools, that have not been identified. Include a timeline under this amendment, 
o If the district lacks capacity to provide all students with an option to transfer to non-identified 

schools, describe the district’s equitable student selection criteria that will provide a transfer option to 
as many students as possible. 
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Abstract 

The School Achievement Indicators Program (SAIF) has been collecting data across Canada on 
13- and 1 6-year-old student achievement in mathematics, in science, and in reading and writing 
since 1993. In 1999, it completed its second assessment cycle and was reviewed in Spring 2000. 
The review design included a survey of officials from all the school boards/districts that 
participated in the science assessment program held in 1999. The results of this study show that 
this stakeholder views as the most pressing issue for SAIP to succeed in its mandate, the need for 
development in four areas: a) Increased teacher and student motivation to participate 
wholeheartedly in the program; b) Effective dissemination options; c) Leadership through 
innovation in teaching and in assessment practices despite high accountability orientation; and d) 
Cost-effective, yet rigorous means of providing both snapshot information and longitudinal 
means of comparisons. Although universally appealing, such approaches have yet to be 
supported by sound educational theory and methodology. 



Since 1993, the School Achievement Indicators Program (SAIP), under the responsibility of the Council of Ministers 
of Education, Canada (CM EC), has been collecting data across Canada regarding 13- and 16-year-old student 
achievement in Mathematics, in Science, and in Reading and Writing. In 1999, it completed its second assessment 
cycle and was reviewed in Spring 2000. The review had three objectives: a) To determine the degree to which the 
CM EC had succeeded in implementing the recommendations adopted from the review of the first round of 
assessments (Crocker, 1997); b) To measure the extent to which SAIP’s objectives, set at the beginning of the second 
cycle, had been attained; and c) To formulate specific recommendations about the SAIP’s aims, operations, and uses. 
To achieve these objectives, the review design included various data collection approaches, one of which was an 
investigation of the perceptions of all the school boards across Canada that participated in the 1999 science 
assessment, toward the national assessment program. The purpose of this paper is to report the results of this survey in 
order to provide researchers and policymakers with a better insight into the general interests, particular views, and 
specific needs of one of the most important stakeholders in large-scale educational assessment programs. 

As a large-scale assessment program, SAIP is similar to the National Educational Assessment Program (NAEP), 
conducted in the United States. Both are national, cyclical programs, administered across regions (states or 
provinces/territories). In both countries, these regions have sole jurisdictional rights over education. Like NAEP, 

SAIP is designed to complement existing assessments in each province and territory. It is essentially a standards- 
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based program focussing on assessment of content and ability via a mixture of multiple-choice, constructed response, 
or short hand-on performance tasks. The program also consists of student, teacher, and school administrator 
questionnaires intended to provide contextual data. Testing usually occurs in May and the final report is made 
available a year later. 

As mentioned above, SAIP is governed by the Council of ministers of education, Canada. The CMEC ’s role in the 
SAIP essentially combines that of the United States National Centre for Education Statistics (NCES), responsible for 
NAEP’s operation and technical aspects, and the National Assessment Governing Board (NAGB), mandated to select 
the subject areas to be assessed and their content framework. In Canada, the CMEC also coordinates national 
participation to other large-scale international testing programs such as the Program for International Student 
Achievement, run by the Organisation for Economic Co-operation and Development (OECD) and the Third 
International Mathematics and Science Study (TIMSS) governed by the International Association for the Evaluation 
of Educational Achievement (lEA). Not all the Canadian jurisdictions participate in the various international 
programs. For a more detailed look at the general CMEC operations, SAIP’s sampling techniques, and the nature of 
the results, see for example the most recent SAIP Report (CMEC, 2000) or the CMEC WEB site. (Note 1) 

SAIP was initially established to meet the following three objectives: a) To set educational priorities and plan 
improvements to curricula; b) To provide the best education to all young Canadians; and c) To report on certain 
indicators to the Canadian public. Despite these noble goals, SAIP’s practices and orientations are geared principally 
towards accountability and overall instructional enhancement rather than improving local teaching, learning, and 
assessment practices. In 1999, all thirteen Canadian jurisdictions, namely the ten provinces and the three territories, 
participated in the Science assessment program. A random sample of students was drawn from each of the 
participating jurisdictions and, within some of these, students were also sampled by linguistic groups (English and 
French). The next four sections respectively present the framework, the methodology, the results, and a discussion of 
the investigation component of the program review. 

Framework 

The theoretical framework for the investigation evolved first from the examination of three documents: The call for 
proposals initiated by the Council of Ministers of Education, Canada, the report on the review of SAIP’s first cycle 
(Crocker, 1997), and the official SAIP memorandum of understanding between the Human Resources and 
Development, Canada and CMEC, 1999). Second, various documents on general program reviews were consulted, 
such as the Standards for Evaluations of Educational Programs, Projects, and Materials, (Joint Committee on 
Standards for Educational Evaluations, 2 ^d ^ 1999). This also included works bn: a) The purpose of a program 

evaluation (Wilde & Sockey, 1 995), b) The role of program reviews (Chelimsky & Shadish, 1 997), c) The use of 
indicators (Posavac & Carey, 1997; Shavelson, McDonnell & Oakes, 1991a, 1991b), and d) The methodological 
approaches to program evaluations (Boulmetis & Dutwin, 1999; Popham, 1 999). Third, the literature review for this 
study considered other actual systematic evaluations of large-scale assessment programs, related models, or proposals 
(Crooks, 1996; Madaus & Pullin, 2000; Ryan, 2002; Shepard, 1977). Finally, the previous experiences of this study’s 
authors in the field of program evaluation also contributed to shaping the framework of the review of SAIP’s second 
cycle (Cousins & Simon, 1993; Macdonell, A., Forgette-Giroux, R., Schmidt, S., Mougeot, Y, & Levesque, J., 1999). 
This led to the development of a framework that essentially consisted of five general areas of questioning on SAIP: a) 
Nature and position among other assessment initiatives, b) Goals and objectives, c) Operations, d) Design, and e) 
Impact. The nature and position focussed on the role, function, relationship, differences, similarities, and linkages 
between SAIP and other large-scale assessment programs. The appropriateness of SAIP’s objectives and the 
identification of possible barriers to attaining these objectives concerned the second theme. SAIP’s operations dealt 
with sampling, standard-setting, grading, and reporting procedures currently in place to meet the stated objectives, 
whereas validity, reliability, and questions that focussed specifically on content and format, nature of data collected, 
and motivational issues, all served to study SAIP’s design qualities. Finally, its impact was examined through 
questions on significance, perceptions, values, and overall influence of SAIP’s results on various educational and 
public settings. 

Methods 

The investigation described here represents only one component of the methodological design for the review and 
targets only one group of stakeholders: School boards (districts or councils) officials. Interviews with jurisdictional 
coordinators or directors were also conducted for the review along with in-depth content analyses of various relevant 
documents. These various components enabled the triangulation of the data that were ultimately reported in an 
aggregated format. The following sections present the subjects, instruments and research design of the survey 
component of the SAIP review. 

Subjects 

All school boards or districts across Canada with one or more schools that participated in the 1999 Science 
assessment program were contacted and invited to participate in the investigation. This meant reaching a total of 412 
school boards from 1 9 jurisdictions: Ten provinces and three territories, with six of these also broken down into two 
linguistic sub-populations at the time of the assessment. Only one member of each board, a senior official, was asked 
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to participate. This official could be a Director of Education, a Superintendent of Education, a designated SAIP school 
Official, a local Coordinator or Liaison person, a Board Consultant, or the Principal of a school that was involved in 
the 1999 SAIP assessment. In some areas, the Director of Education is the highest ranking official while in others it is 
the Superintendent of Education. When both are found within a school board, the Director of Education usually has 
priority. 

Instrument 

A written questionnaire was developed based on the framework’s five themes. A first draft was submitted for 
validation to two experts familiar with SAIP, one French-speaking and one English-speaking. Their task was to 
determine the relevancy, comprehensiveness, and linguistic equivalence of the questionnaire. The final version 
included seven general information questions and six attitudinal questions with 73 sub-questions. Twelve questions 
offered multiple options while two were open-ended although all questions provided space for additional comments. 
The open-ended question # 1 3, for example, asked “What suggestions would you offer regarding SAIP and its various 
components in order that your school board or district gives it high priority among all assessment initiatives? Of 
particular interest however were questions 9 and 10. Question 9 asked :“To what extent do you agree with the 
following actual SAIP parameters?”, while Question 10 read as: “To what extent do you agree with the following 
proposedSKl? parameters?” The proposed parameters emerged from comments made by the SAIP’s designated 
jurisdictional coordinators who had previously participated in the semi-structured interviews, from the 
recommendations that resulted from the review of SAIP’s first cycle, and from their relevance with respect to general 
school boards’ interest and needs. 

Procedure 

The questionnaires were sent to the top administrator of each of the 412 school boards. The senior officials were 
instructed to fill out the questionnaire themselves or to forward it to someone from the school board that had been 
actively involved in the 1999 science assessment. The questionnaires were distributed during the last week of May 
2000 with specific written instructions to return the completed questionnaires in a self-addressed envelope by June 30, 
2000. Two sets of follow-up telephone calls were made by a superintendent to selected school boards from each of the 
jurisdictions and their sub-population in order to achieve a maximum rate of return by all sampled populations. The 
first was conducted in the second week of June 2000 to approximately half of the boards within each jurisdiction to 
see whether they had received the questionnaires. The next series of calls were made in August to approximately 
twenty randomly selected boards from each jurisdiction that had a return rate below 30 % by the end of July. 

Results 

In all, 147 questionnaires were completed and returned, yielding an overall response rate of 36%. Of the 19 
populations, four had all their participating schools in a single school board. Two of these four boards returned the 
completed questionnaire. Closer examination of the distribution of responses by jurisdictions reveals that the nine 
“smaller” jurisdictions and minority groups, that is those that had from two to 12 school boards, yielded response 
rates varying from 33% to 100 %, with a mean of 59% and median of 54%. The six “larger” jurisdictions, namely 
those with over 12 school boards (actually between 22 to 62), gave response rates varying from 27% to 36 %, with a 
mean of 32 % and a median of 33%. The rate of response for the 1 5 jurisdictions with two or more school boards 
therefore ranged from 27% to 100%, with an average of 48% and a median of 46%. It can also be reasonably stated 
that respondents were drawn from both small and large school boards and districts, from both remote and rural 
settings, from both central and urban areas, and from both the majority and minority linguistic groups. 



Most questionnaires (80%) were completed by Directors of education. Superintendents or Board consultants. Eighty- 
six percent of respondents said they were more or less familiar with SAIP or knew it well. Respondents participated 
mainly in the coordination of the study, the test administration, and the communication of results. Although most 
questions included a four-point rating scale, dichotomized results (e.g. totally agree and agree versus more or less 
agree with disagree) are reported here with respect to each of the five themes. 

Nature 

The specific information on this topic was provided mainly in the two open-ended questions. Respondents generally 
recognize that SAIP provides a valuable index from a national perspective but state that it should distinguish itself 
from international and provincial initiatives by: a) Highlighting cross-curricular competencies if sampling remains 
age-based; b) Introducing innovative teaching, learning, and assessment approaches that are applicable to the 
classroom; and c) Adopting a diagnostic and interpretative approach to contextual and achievement data. 
Approximately 70 % of respondents also suggest that secondary analyses be performed on the data collected. 

Goals 



o 

ERIC 



Although indicating being familiar with SAIP, three quarters of respondents (75 %) also admit that they are somewhat 
or not very aware of its stated goals and objectives. With respect to the appropriateness of the objectives, they 
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generally suggest the need for an objective that would implicate some form of comparison of results. The rank order 
of the percentage of respondents in agreement with the various types of comparisons of results is presented in Table 1 . 

Table 1 

Rank Order of Percentages of Respondents in Agreement with 
Types of Comparisons of Results 



Longitudinal comparisons from one assessment cycle to next 


87% 


With national results (averages) 


81% 


With national expectations (standards) 


65% 


Between the two age groups (13 and 16) 


64% 


With local expectations 


50% 


Among jurisdiction results 


44% 



Table 1 shows support from school board officials for comparison of data from one assessment cycle to the next and 
with the national averages. 

Operations 

Results under operations are reported in terms of SAIP’s general administrative, sampling, expectation/standard- 
setting, scoring, and reporting procedures. Eighty-nine percent (89 %) of respondents agree with the statement that 
SAIP meets the given time-lines and 67% believe that SAIP is relatively easy to administer. It is interesting to note 
that only 51 % of the respondents favour the month of May as the best time for test administration, 68 % disagree 
with conducting these in February or March, and 77 % disagree with a Fall administration, thus failing to achieve a 
consensus on the best time of year for administrating the various assessment programs. Three quarters of respondents 
favor sampling of 13- and 16-year-old students rather than sampling grades. Two thirds agree with comparisons of 
results with expectations or standards and generally approve of the five-point rating approach to scoring. Finally, 
nearly 70 % of respondents support the idea of disseminating some form of school board, school, or individual-based 
results for motivational purposes. 

Design 

Questions on SAIP’s design asked for level of agreement with the assessment’s focus on collecting disciplinary and 
contextual data as well as from theoretical and practical aspects of achievement data gathering techniques within 
Mathematics and Science. They also looked at motivational issues and perceptions around assessment cycles. Results 
show that 77 % of respondents agree with the collection of contextual data. Most concur with SAIP’s mandate to 
conduct assessments in Mathematics, in Science, and in Reading and Writing and over 85 % favor both the theoretical 
and practical components of the science and mathematics assessments. However, 66 % of the respondents state that 
SAIP does not sufficiently encourage or motivate students to give their optimal performance, thus addressing validity 
and reliability concerns. Finally, nearly 80 % of respondents support the three-year assessment cycle within a 
discipline. 

Impact 

Eighty three percent (83 %) of the respondents believe that the program has little or no positive impact on assessment 
practices in the classroom and approximately three quarter of respondents say that SAIP has little positive influence 
on setting priorities in the various disciplines assessed, on curricula, on public perception of the quality of education, 
on research initiatives, and on teaching practices. Approximately 85 % of respondents, however, would recommend 
those school boards that have not yet participated to the various SAIP programs to do so. 

Discussion and Conclusion 

Despite the overall low response rate obtained in this study, the results are telling. With one exception, the higher 
response rates are provided by those jurisdictions or minority populations with less than 12 school boards. Although 
many jurisdictions take part in other large-scale assessment program such as TIMSS and PISA, most respondents 
indicate general support for the SAIP because it is the only national long-term assessment program in which all 
thirteen educational jurisdictions and respective linguistic sub-populations participate. They also appreciate the 
complementarity of SAIP’s results to those obtained through regional and other international assessment programs. 
Respondents stress, however, that SAIP should be forward looking and be more than a simple indicator system 
serving accountability purposes. Their responses lead toward interesting suggestions for each of the five themes. 
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These are discussed in the following sections. 



Nature, role, and position 

With respect to the nature, role, and position of SAIP, the respondents value the nationally representative and 
continuing aspects of SAIP. Given that participation is voluntary, however, it becomes important that SAIP continues 
to be attractive and relevant to this stakeholder, particularly to those school boards that have sufficient resources to 
implement their own assessment program or to participate in most international ones. In that sense, SAIP should 
consider three suggestions offered by the school boards. First, they propose that if SAIP remains an age-based 
program as opposed to grade-based, then it should go beyond the assessment of disciplinary contents and basic skills 
in order to focus on socially relevant general competencies such as critical thinking, information management, 
speaking skills, and on attitudes such as civic values, self-awareness, self-esteem, and student engagement (Jones, 
2001). Major organizations, such as the IE A and the OECD, already provide leadership in the assessment of such 
competencies via their own large-scale programs. At the onset, this option may appear to be duplicating efforts and 
resources given that many jurisdictions also participate in these major international assessment programs but, so far, 
SAIP has had the advantage of involving all jurisdictions and of being better able to respond to national educational 
concerns. 



Second, as mentioned above, the respondents wish to see SAIP as more than an indicator program, one that would 
offer specific perspectives for adjustment and intervention with respect to setting educational priorities, targeting 
curricular improvements, highlighting best teaching strategies, and fostering innovative assessment practices. This 
push for a shift from an indicator system, i.e., one that provides information that can be used to improve education, to 
a monitoring one, in other words, one that further analyses and interprets key contextual and achievement data in 
order to propose prescriptive feedback, stems largely from those smaller jurisdictions with scarce resources. The 
resulting feedback could subsequently translate into the introduction of innovative teaching, learning, and assessment 
approaches that are applicable to the classroom. This last statement relates to the respondents’ third request. 



These three requests contributed to the formulation of two specific recommendations in the review: a) That the 
concept of achievement indicator be broaden to include assessment of general competencies as defined by the pan- 
Canadian public and b) That SAIP be assigned a diagnostic function with interpretation of the most obvious links 
between contextual data and achievement (Forgette-Giroux & Simon, 2000). These recommendations are rather 
demanding. As with other large-scale studies, SAIP must re-examine it priorities with respect to accountability versus 
instructional orientations toward educational improvement (Popham, 1999) and eventually aim at establishing a 
balance between the two goals. So far SAIP has been mainly oriented toward the need for greater accountability. 
Moreover, it must rely on sufficiently sound theory or design to efficiently explore any relationships among 
background variables and achievement, to claim causal inferences or to explain why the comparison of certain groups 
yields different results (Bechger, van den Wittenboer, Hox, & De Glopper, 1 999). 

Goals and objectives 

Although SAIP’s present objectives are ambitious, universal, and aim at continually moving targets, thus ensuring 
their enduring validity, respondents wish to add another objective concerning the specific comparison of results from 
one assessment cycle to the next and with national data. This implies that SAIP should be given the dual goal of 
providing a snapshot of current achievement levels across jurisdictions and of measuring progress over time. 

However, in addition to the lack of proper theory to explain comparative differences in achievement as mentioned 
above, such an objective can also create tensions such as those experience throughout NAEP’s history, because a 
single assessment system cannot adequately serve such diverse purposes, each with its own set of assumptions, 
processes, and consequences (Linn, 2000). For example, longitudinal programs must find ways to rely on stable and 
reusable instruments while remaining fully aligned with national standards, actual curriculum contents, and evolving 
theories. In order to meet these challenges, NAEP presently operates two systems, the Main NAEP for longitudinal 
measures, and the State NAEP for cross-state comparisons. A recent NAEP review, however, has called for 
streamlining and for merging some aspects of the two programs (Pellegrino, Jones & Mitchell, 1999). If the CMEC 
decided to stress pan-Canadian longitudinal comparisons from one cycle to the next and comparisons with national 
averages, then such a decision would have major consequences on the entire program’s structure and development 
(Bechger, et al., 1999). SAIP is currently not designed to meet such goals and does not have the resources to conduct 
parallel systems. As with all large-scale studies of achievement, if SAIP decided to opt for the comparative route, then 
it would have to meet at least three conditions to ensure the comparative validity of its results: Construct equivalence, 
scale equivalence, and measurement equivalence (Bechger, et al., 1999). So far, these equivalencies are unattainable 
mainly because the theoretical frameworks underlying the most universally accepted competencies such as reading, 
problem solving, and scientific reasoning are constantly evolving, performance scores across time and groups are 
interpreted using arbitrary scales, and scale equivalence is greatly hampered by issues such as test translation. Perhaps 
the only way out at the present time is to document and make all related information as comprehensive and 
transparent as possible in order to arrive at the most valid interpretations when comparing results. 

Operations 

Respondents generally agree with the existing parameters around SAIP’s administrative, sampling, 
expectation/standard setting, scoring, and reporting procedures. Some of their comments, however, stress greater 
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participation of various school board personnel in all stages of SAIP’s assessment programs. Similar statements are 
expressed in scholarly papers such as that of Hunter and Gambell (2000), and with respect to other large-scale studies 
such as New Zealand’s National Assessment Monitoring Project (NEMP) (Flockton & Crooks, 1997). They claim 
greater empowerment and satisfaction by teachers and by other school board members who participate in substantial 
and independent training sessions and in the actual implementation of various aspects of the assessment program. 
Moreover, respondents point toward operational changes that consider greater teacher involvement and teacher input. 
As a result, the SAIP review offered a recommendation stating that CM EC implement, for example, a two-step 
standard-setting process in which the first series of expectations be formulated principally by teacher representatives. 



Another operational issue that has not been overtly stated by the respondents but that was raised indirectly through 
their responses is the debate over grade- or age-based sampling. Although age-based sampling is seen as a more 
probabilistic in theory than a grade-based approach, in practice, it is not always respected. Local SAIP coordinators 
experience difficulty in implementing the recommended student sampling process at the school level (Forgette-Giroux 
& Simon, 2000). This is probably due to the fact that practitioners tend to view a classroom as a unit and thus see age- 
based sampling as causing significant disruptions to the class. As a result, student sampling is not done uniformly 
because of a conscious or unconscious need to minimize those perceived disadvantages. Although it is current 
knowledge that, as a result, most schools therefore select students based on their own criteria, unfortunately these 
practices are not always documented. School sampling is also a problem in jurisdictions with smaller populations 
because of over-sampling, which means that the same schools participate repeatedly in large-scale programs, again 
seen by many practitioners as causing significant disturbance. 

Design 

This section dealt with the format, content, and measurement qualities of the assessments. Sound decisions about 
policies and the allocation of resources depend on quality data. Although a significant number of respondents support 
most existing design parameters, they acknowledge the need for greater willingness by teachers and students to fully 
engage in the program in order to increase the validity of the data. Despite the respondents’ understanding that the 
program is not suitable to offer school-based or individual achievement results, their comments point toward the need 
to publish some type of school -based information, such as thematic reports, that would highlight current exemplary 
assessment practices. Increased teacher participation and support without added stress, time, and effort are also seen 
as a means for attracting teachers to further embrace the process and to encourage their students to provide optimal 
performances. 



To that effect, the SAIP’s review thus recommended the exploration of incorporating local assessment practices 
within the current SAIP design. This give-and-take approach is expected to further empower and meet the needs and 
satisfaction of participating teachers. However, this is easier said than done and the issue of motivation has been 
raised in other large-scale studies (Hattie, Jaeger, & Bond, 1999; Lane & Stone, 2002; Wilson, 1999). Within the 
SAIP’s context, external motivation, i.e. students are motivated if teachers are motivated, plays an important role. 
Offering pizza to participating students, as several jurisdictions do, is not enough to entice students to invest 
wholeheartedly in the assessment. The debate around greater teacher and student involvement leads to several 
meaningful research questions such as: Can such a low-stakes program provide sufficient relevant information to 
schools to attract teachers and their students to such assessment programs? To what extent would the incorporation of 
innovative classroom-friendly assessment practices of socially relevant competencies entice teachers and their 
students to fully collaborate? What are some of the most successful practices to motivate students and their teachers to 
provide optimal performances within age-based sampling programs? What type of feedback would promote greater 
participation by the teachers and their students? 

Significance 

In terms of impact, the data indicate that a large proportion of respondents believe that SAIP has little or no influence 
on various educational contexts at a local level. In that respect, CMEC has not succeeded in implementing some of the 
recommendations from the first SAIP review, particularly those addressing a) better dissemination of results at local 
level, b) increased SAIP visibility, and c) greater ownership of SAIP’s objectives by school-board administrators. 

This implies a definite need to develop an awareness-building plan that aims specifically at school-based 
stakeholders. Such an issue can be addressed through various approaches, namely: a) Elaboration of reports for 
different audiences, b) Publication of clear and pertinent frameworks that reflect the most recent theories, research, 
and practice (e.g., Campbell, Kelley, Mullis, Martin & Sainsbury, 2001; College Board, 1999; CCSSO, 1999a, 1999b; 
Flockton & Crooks, 1997), c) Release of actual and practice items and tasks, along with their respective scoring 
guides to serve a innovative practices (e.g., Robitaille, Beaton & Plomp, 2000), d) Development of a web site that is 
continually upgraded and maintained, e) Support for secondary data analysis studies, and f) Linkages of SAIP results 
with those from provincial and international assessment programs. NAEP, TIMSS, and PIRLS have been known for 
their sustained effort to inform their various stakeholders and audiences in a timely fashion and through a variety of 
reports. NAEP, for example, publishes the following: Report cards, Highlights reports, Instructional reports, State 
reports. Technical reports. Cross-state reports, Trend reports. Focussed reports^ and Service reports, and its 
dissemination process is continually examined to improve the usefulness of these various reporting formats (Horkay, 
1999). The review of SAIP’s second cycle has led to the formulation of several recommendations to that effect, 
particularly with respect to the release of sound and clearly articulated frameworks, exemplary items and scoring 
guides, and secondary analyses of relationships among contextual and achievement data. 
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Many of the jurisdictions that participate in SAIP also join other international large-scale assessment programs and 
most large populations, defined in terms of number of school boards or districts, implement their own. Eleven 
Canadian jurisdictions took part in the last TIMSS-R assessment and were involved in the PISA, 2000 program. The 
survey did not question the practitioners on SAIP’s merits in relation to alternative programs because of the 
jurisdictions’ varying degree of involvement in these. However, such a question was formally asked to each 
jurisdictional coordinator or representative. Their answers generally indicated that these programs provide valuable 
complementary data but they do not have a confirmed continuous cyclical plan and tend to provide a perspective that 
is more or less detached from the national educational concerns. With its planned long-term assessment cycles, SAIP 
can carve out an enviable and enduring place among large-scale evaluations by considering the following three 
mandates: a) Focussing on cross-curricular competencies that are valued by the Canadian public, b) Adopting a 
diagnostic function aimed at offering innovative perspectives for adjustment and intervention in educational priorities, 
in curricula, and in teaching, assessment and learning strategies, and c) Increasing its usefulness toward the smaller 
size jurisdictions or minority populations. 

In conclusion, this study’s objective was to report the voice of one major stakeholder: The school boards. For a 
variety of reasons, such as the timing of the investigation, a major overhaul of the educational system, a considerable 
employment turnover, and responsibility overload, fewer respondents than anticipated completed the survey 
questionnaire. Nevertheless, this voice is a fundamental one to which any large-scale program should pay particular 
attention given its frontline position. In other words, it is the one that provides the raw data. In future reviews of large- 
scale assessment programs, this stakeholder should therefore be consulted through methodologies offering further in- 
depth and relevant prompting of some of the major concerns expressed in this study. This is especially true if the 
instructional enhancement is as much a priority as is educational accountability (Popham, 1999). Such methodological 
approaches could perhaps include focus group sessions in which directors, superintendents, principals, local 
coordinators, board consultants, and teachers would share and confront their views regarding those aspects of the 
assessment programs that have the most impact on them (Haertel, 2002). As this study results show, the most pressing 
issues to be debated would likely include the following: a) Increase teacher and student motivation to participate 
wholeheartedly in the process, b) Develop effective dissemination options, c) Identify ways to ensure that the 
assessment program can continue to provide leadership through innovation in teaching and in assessment practices, 
and d) Finding cost-effective, yet rigorous means of simultaneously providing snapshot information and longitudinal 
means of comparisons. Although universally appealing, such approaches have yet to be supported by sound 
educational theory and methodology. 

Finally, despite the fact that a large proportion of respondents in this study viewed SAIP as having little impact on 
various educational contexts and were more or less aware of its objectives, most indicated that they would 
recommend other school boards to participate in SAIP. It appears that for many, SAIP has effectively carved out an 
important place for itself among large-scale assessment programs because of its pan-Canadian nature, its capacity to 
involve all jurisdictions, and its ability to respect many of the technical requirements of such initiatives (Forgette- 
Giroux & Simon, 2000). Given the voluntary aspect of this participation and the importance of gathering valid data, 
however, SAIP should invest in maintaining its leadership role by raising its instructional priority, by increasing its 
visibility, and by inspiring local policymakers, administrators, teachers, and students through attractive and 
meaningful ways. 
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Abstract 

It is not surprising that private education is gaining importance in China given the overall context 
of huge national efforts toward building up a “socialist market economy.” However, the fast 
growth rate in both the quantities and the qualities of profitable private schools in a socialist 
society is beyond what people usually expect. This paper looked into the modem history of 
private education in China and found that such a huge resurgence of private education is rooted 
in the heritage of private education in the Chinese society. Private schools were the precursor of 
modem Chinese education. They played an important role in the country for most of the time. 
When the government policy became more flexible and household income increased 
substantially, such a heritage revived and becomes a stimulating factor in the education sector. 



1. Introduction 

The huge population base of China creates the largest education industry in the world. 1999 figure shows a student body 
of 320 million, accounting for about 30% of world’s student population (Qu, 2001). Limited financial resources severely 
restricted educational development in China. With lower-than-average national effort and fiscal effort measures (Tsang, 
2000b), the number of Junior high school classes that accommodate more than 66 students has increased from 61,000 in 
!993 to the recent 132,000 (Jiang, 2001), the higher education gross enrollment ratio (for age 1 8 — 22) was only 10.5% 
in 1999, and only 1% of labor force has received 4-year and above tertiary training (Qu, 2001). In order to realize the 
developmental goal of 2010 (Note 1), the Chinese government launched a reform of the structure, administration and 
financing of education in 1985, decentralizing and diversifying the providing and financing of education (Tsang, 1993). 
Non-governmental schools revived after the 1985 policy in respond to both the excess and differentiated demand. 



This is not the first time in history that China has a private education system. With rapid growth in the previous two 
decades, a lot of problems emerge on such topics as legal status of private schools, their legislation, ownership and 
quality. Review of available literature shows policy makers are more concerned with the commercial potentials of 
private schools than educational quality and effectiveness. The retrospection of contemporary private education tells 
people there are goals other than business profit in educational development, and that education is about devotion, truth 
and innovation. This paper records the contemporary private school history in two parts: pre-1949 era and post- 1949 
era. 
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2. Pre-1949 era 



Though the tradition of private education dates back to thousands of years ago with Confucius, and for a long period it 
was the dominant form of education, the modem private school system (or “new school system”) did not appear till 
around the 1 840. 

1840 is the year that is usually used in Chinese history books as the beginning of the contemporary period, when China 
was defeated in the Opium War and began its journey of being colonized by western powers. Before this, China had 
been such a self-contained nation that it was reluctant to open to the world. Opium War was the first attempt to force 
open the door to China. Yet even such a shameful and additive commodity yielded to the strong resistance of Chinese 
people. War became the final choice. 

The door was open. Through a series of humiliating defeats after 1840, the blind arrogance of the Chinese was shattered, 
and they began to accept, with great agony, the backwardness in their national development level. Various endeavors 
were made to save the nation. Education modernization is among the most important attempts. 



In this education reform movement, private education played a crucial, if not the dominant role. Not only did private, 
modem schools appear earlier than governmental schools, but also that they were generally of better quality at the 
higher education level. Schools in this period can be divided into four types: Government-owned schools, mission 
schools, schools run by Chinese and sishu. The latter three types are private schools. However, the last one, sishu 
(which, translated literally, means private school), does not belong to the modem school type. Yet it persisted in modem 
China till the early 1950s (Deng, 1997). The development of modem school system can be summarized in the simplified 
diagram below. 



1840 Mission Schools 1952 



1862 Modern Government Schools 



1904 Schools Run by Chinese Educator 



Old-fashioned Sishu 



2.1 Mission schools 



The earliest modem private schools were founded by missionaries. Its development in China can be divided into three 
periods: 184(>— 1904, 1904— 1925, and 1925—1949. 

The starting period (1840—1904). As the starting of a series of “unequal treaties” between the Qing government and the 
Western powers, “Nanjin Treaty” (1 840) knocked open the door of China by setting up “trading ports” in Eastern China 
and helping the Church obtain legitimate status to spread Christian Gospel in those cities. Previous experience had 
shown that with the deeply entrenched Confucius tradition, Chinese were resistant to Christian beliefs. The spread of the 
Western religion had proven to be unsuccessful. For example, it took the American Methodist mission in Guangzhou ten 
years to make the first Chinese convert (Deng, 1997). To attract more conversions, missionaries tried to set up hospitals, 
schools and printing agencies so as to win the support of the Chinese people. Among these agencies, schools did more 
than just helping people and winning their hearts: They also passed on Christian beliefs directly. With the subsequent 
“unequal treaties” (Wangxia Treaty in 1844, Huangpu Treaty in 1844, Tianjin Treaty in 1858, etc.). Western powers 
further obtained the rights to spread Christian Gospel in the interior areas of China. Religion in company with mission 
schools penetrated into central China. 



Mission schools in this period had several common characteristics. First, their scale was quite small. For example, the 
first missionary school set up in Macao (1839) admitted only 6 children for the first year. Second, the establishment and 
administration of these schools were not systematically supported by the missionary organizations in their home 
countries, as was the case in the later periods. Schools in this period usually did not have any administrative staff 
Decisions on curricula and teacher recruiting were left to one or several missionaries. Third, not unexpectedly, with 
limited capability of the personnel and funding, schools in this period were mainly primary and secondary schools. 
Fourth, mission schools were still not completely accepted and identified with by society, both on the intelligentsia level 
and the grassroots’ level. Due to the long isolation of China from the rest of the world, foreigners were looked as 
barbarians by many people. This severely restricted the development of mission schools. Fifth, this negative perception 
toward foreigners/mission schools reduced their appeal to students. As early schools provided free accommodation and 
food for the student, poor and homeless children attended the schools run by “barbarians” only to remain alive. This is a 
sharp contrast with the situation in latter periods of development. Finally, the curricula mainly included the Christian 
Gospel and a small amount of rudimentary knowledge of reading and calculating. 
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The number of missionary schools in this period was quite small. For example, Deng (1997) reported that the First 
General Conference of Protestant Missionaries in 1877 reported only twenty Protestant mission schools with 231 
students. Among these schools include Chongxin yishu (1844), Menyang School (1864), Wenhua School (1872), 
Fanwandu English Language School (1879) and Huiwen School (1889). 

1904 — 1925, Modem governmental schools did not appear till 1862, when Jinshi Tongwen Academy was established. 
After a series of defeats in the battlefield, some Qing officials realized the importance of technology and foreign 
language. The government split into two groups in the late 19^^ century. One group strongly supported the learning of 
Western technologies and military strategies. The other advocated political and educational reform within the old 
government system. Both groups belonged to the feudalistic government, hoping to save the nation without 
overthrowing the feudalistic system (Chen, 1982). As a result, in the field of education, three types of new schools 
appeared: military schools, language schools and technology schools. They were modeled mainly after the early mission 
schools. 

On the other hand, outside the government, one of the largest farmers revolts, ‘‘Peaceful Paradise” {taiping tianguo), 
lasted for decades and controlled in half of the provinces across the country. This movement started under the name of 
God and claimed to be a religious movement. Later, this movement was quenched by the Qing government with some 
help from foreign military forces. 



Western influence had penetrated China both within the government and without by the latter half of the 19th century. 
This created a nourishing environment for mission schools to expand rapidly (Deng, 1997). The expediting expansion 
coincided with other issues that caused great social change. The first was the Boxer Rebellion. This was organized by 
farmers in 1900. It fought directly against the Western powers, including the evangelistic activities performed by 
missionaries. This movement stunned the Western nations with such gravity that they began to realize the importance of 
controlling a people by influencing its mind first. Education became viewed as one of the most efficient ways of 
affecting people’s minds and gained strong support from the Western nations. Through education, missionaries 
attempted to win support from the intelligentsia in the hope of making them the future leaders of China, and vehicles for 
control of the country. Compared with the schools in the previous period, missionary schools were supported by various 
religious organizations and foundations, including the Rockefeller Foundation. 



Another issue that changed the developmental context for missionary schools (and other forms of private education as 
well) was the implementation of the ''Guimao Education System” in 1904. Many reformists became active in politics 
around 1900. Some of them saw the annihilation of the old education and examination system and the establishment of a 
modern school system as the only way out of the dire condition of the nation. These reformists made several attempts to 
improve the nation within the framework of existing government. The most famous attempt was the Wuxu Legislature 
Reform (“ITmlxm” refers to the year of 1898 as expressed in the traditional Chinese calendar). Among the reform 
package, attempts were made to change the educational system. For example, it set up “Jinshi Daxuetang”, which is the 
embryo of today’s Peking University. The reform also suggested abandoning the traditional examination system, adding 
political and economic studies into the curriculum, and setting up professional schools in the field of law, finance and 
diplomacy. The reform is famous for its transience: It remained effective for only around 100 days. Almost all the 
contents of that reform were abandoned. Nevertheless, the Qing government preserved “Jinshi Daxuetang”. 



Although the “Wuxu Legislature Reform” was aborted, the reformist ideas began to spread in society. In 1903, the Qing 
government proclaimed the new Royal Regulations for Schooling {Zouding Xuetang Zhangcheng). In this regulation, a 
new school system was designed and the local government on all levels was required to establish new schools according 
to the regulation. Institutions were set up to make sure students got the compulsory education, and governors were 
requested to assist the proliferation of such education (Deng, 1997). Shortly after the proclamation of this new 
regulation, a ministry of education was established, and the old examination system {keju, which required the writing of 
an essay, often on politics or morality in the form of an “eight-legged” essay (Note 2) ) was finally abolished. With this 
great ambition to completely rebuild the national education system, together with the dearth of governmental funding 
for public education, private education, including both mission schools and private schools run by Chinese, faced 
excellent possibilities for growth. 

Missionary education in this period featured a rapid expansion in terms of the number of schools. According to Deng’s 
(1997) description: 



In 1899 there were 1,296 Protestant missions in the Celestial Empire. By 1914, the number had more 
than quadrupled. By 1906, there were over 2,000 elementary schools and 400 middle schools run by 
Western missions. On the eve of the Republican Revolution of 191 1, Western missionaries operated 
3,145 schools in the country. Protestant mission schools enrolled 138,937 students in 1912. Catholic 
and Orthodox mission schools had a student body of 50,000 to 100,000. In Southern Manchuria, the 
Japanese ran 28 schools with a total enrollment of 5,55 1 students (Deng, 1997:32) 

In addition to the development in numbers, mission schools scored rapid growth in higher education. Many new 
colleges and universities were established. Even some secondary mission schools were upgraded or merged into 
colleges. Examples include Jinlin University of Nanjing (191 1), Jinlin College of Women (1915), Wuchang University 
(1910), Shangdong Christian College (1902), and Qilu University (1915). Most of the top-tier universities in China 
today were established in this period by missionary organizations. The most outstanding examples are Xiehe (Concord) 
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Medical School and Tsinghua University (Hu, 1994), the former being the best medical school and the latter one of the 
best comprehensive universities in China today. 

There were several trends that characterize education development in this period. First, out of the necessity of 
identifying themselves with the Chinese elitist class and becoming integrated into the largely secular society of China, 
missionary organizations gradually shifted their emphasis from religious dissemination to knowledge education. 

Mission schools gradually became general educational institutions. As a result, missionary organizations made great 
efforts to improve their academic quality at the same time as they expanded the quantity of schools. These organizations 
attracted well-trained teachers with high salaries, extended the length of academic programs and improved the 
curriculum design. Second, with more emphasis on general education, vocational and professional trainings were also 
gaining ground. For example. Saint John University established its own medical department in 1 896, while the Concord 
Medical School of North China was founded in 1905 under the auspices of the Rockefeller Foundation. Nursing and 
dental training were also offered in these medical schools. Third, the student body of mission schools changed. Focusing 
more on education itself than on religion, schools implemented entrance examinations to ensure student quality (Hu, 
1994). As mentioned before, the student body in the first phase of development was mainly composed of children from 
poor and religious families. In this period, however, the weakening emphasis on religion, the stricter admission control 
and better facilities attracted students from the rich and intellectual families. Starting about 1910, it became increasingly 
fashionable for affluent Chinese families to send their children to mission schools (Deng, 1997). 

With the above changes in mission education, the source of funding for mission schools also changed, the fourth 
characteristic of this period. There were two principal sources of funding. The most important one was the support from 
the mission organizations and foundations from the Western countries. What makes this source different from the 
previous period is “mission societies began to break sectarian boundaries to pool their resources together in setting up 
and running Christian colleges” (Deng, 1997, p.33). For example, the above-mentioned Qilu University was jointly 
supported by the American Presbyterian and the English Baptist missions (Deng, 1997). The second source of funding 
was tuition. This was new to mission education in China. In the first phase, mission schools had to offer free 
accommodation, free education and free traveling to attract students. With more affluent students joining the mission 
schools and the increasing popularity, mission schools in this phase began to charge tuition and gradually became a 
luxury for rich families. As a result, tuition accounted for a significant proportion of mission school financial resources. 
During the 20s and 30s, one third to one half of the funding came from tuition in some large-scale universities in Eastern 
China (Hu, 1994). 



From 1904 to 1925, China transformed from a feudalistic society to a Nationalist state. After the revolution in 1911 that 
overthrew the Qing Dynasty, China went through rapid economic growth. Deng (1994, p.39) reported an annual 
industrial growth rate of 13.4 percent between 1912 and 1920.The emerging new society generated a great national 
demand for education. With the government encouraging private contributions and investment in education, private 
schools, including mission schools, achieved a rapid, and sometimes rampant growth. During this period, western 
mission schools were not required to register with the Chinese government. Exemption of registration means the 
Chinese government had no control over those schools. Mission schools were registered within their native countries. A 
trustee board was usually set up in the native country to control all the administrative, financial and personnel issues 
(Hu, 1994). This last characteristic of the second phase, together with other social and international environment change, 
led to a new stage of mission school development in China. 

i925 — 1949, The end of the First World War in the previous phase inspired the national awareness of independence. It 
aroused the nationalism in two ways. First, the War attracted the attention and energy of Western powers to the 
European theatres. This left a precious chance for the national industries of China to develop and compete with foreign 
enterprises. As a result, China achieved rapid economic growth during the period. Second, although China did not gain 
benefits from the treaty following the war, it was listed as one of the “winning” nations. This helped its people regain 
some confidence and pride in their own country. With the establishment of a nationalist country in 1925, this 
nationalism was further strengthened. Chinese intellectuals began their quest for a new national identity. Under the 
influences from other countries, many practices were borrowed and adapted to the special context of Chinese society. 
Some people held a strong faith in democracy and science, while others sought national salvation through Marxism and 
Leninism, which had scored successes in the experiment in Russia. Students returning from overseas brought back new 
ideas with them. Tao Xingzhi and Jiang Menglin, two educators who had graduated from Teachers College, Columbia 
University, were among the most prominent examples who sought national identity through education experiments. 



In such a context, mission education was facing serious and widespread criticism. It was considered by young 
intellectuals as a denationalizing force and an imperialistic and colonizing agency of western powers. Mission education 
was also regarded as a religious propaganda agent. In October 1924, all these “evil effects” of mission education were 
formally included in Resolution VII of the National Federation of Provincial Educational Associations Annual Meeting. 
The resolution also recommended that mission schools: should register with the government; be supervised by local 
committees; require their teachers to possess teaching qualifications; set tuition no higher than other private schools in 
the same district or province; and should cease propagating religion (see Chinese Christian Education, 1925). The 1925 
Chinese Christian Education Conference in New York also reported that 

In spite of the fact that both government and private schools have been crippled for finances, and that 
the whole country has been unsettled, .. .[there is] growing sense of confidence on the part of the 
Chinese educators with reference to their ability to develop a sound educational system for their 
country. (1925, p. 14) 
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The May 30 Massacre in 1925, starting with the killing of one Chinese worker in a British textile plant in Shanghai, 
ignited an anti-imperialism protest that swept across the nation. Mission schools, mainly colleges, experienced their 
most difficult time in China. In the following year, registration began to be officially required for mission schools. It 
was also ruled that the presidents and administrative staff should be Chinese. Religious classes and practice were 
changed into elective activities and mission schools could not force any student to participate. 

In such a stringent environment, mission schools became localized, secularized and more academically oriented. 

Chinese teachers were employed, and many western administrative staffs were replaced by Chinese intellectuals. 

Classes began to be taught in Chinese. And with more emphasis placed on academic quality, some mission colleges later 
boasted the strongest and best research in some majors like Chinese language and culture. 

According to Deng (1997), Tatsuro & Sumiko Yamamoto (1953) reported that three thousand missionaries had left 
China by the end of 1927. While mission higher education was undergoing personnel shifts and re-design, Christian 
elementary and secondary schools also decreased in numbers. In fact, the situation of mission education continued to 
deteriorate after Guo Ming Dang (Nationalist Party) ended the ruthless battles between warlords and unified the nation 
in 1928. As a ruling party, Guo Ming Dang tried to justify its governance through its control over education. Similar to 
today’s practice of the Communist Party, party representatives were dispatched to all schools, including mission 
schools. In 1933, Western missions were forbidden to run elementary schools for Chinese children (Deng, 1997). 



After the invasion of Japanese troops first in 1931 and later intensified in 1937, the entire education system in China 
experienced huge losses. Most of the schools retreated to the inland areas. Mission schools also suffered from the 
chaotic situation. Starting from the late 30s, the government began to offer small amounts of financial support to private 
schools, including mission schools (Hu, 1994). This support did not come free. Governmental support changed some 
private schools into public ones. Shortly after the end of WW II in 1945, the civil war broke out, leaving private schools 
no time for recovery. Since then, the fate of mission schools, as well as other private schools, has been decided when the 
Communist Party took over the government in 1949. By the mid 1950s, all private schools have been transformed into 
public ones. 



Comments on Mission Schools in the Contemporary History of China. Mission schools have been regarded as one of the 
great manifestations of Western colonization of China. Bringing with them brand new and often contradictory 
ideologies, mission schools clashed with the traditional culture dominated by Confucian ethics. Chinese society was 
forced to accept Western influence with agony and humiliation. At the same time. Western institutions helped Chinese 
people to see more choices and alternatives in every aspect of social life. 



Mission schools invaded Chinese culture in a more insidious fashion than physical invasion. Missionaries and Chinese 
people interacted with each other. Mission education was gradually but never completely accepted by the intellectual 
society. The development of mission education in China can either be described as a cultural communication, or a 
cultural invasion since there was no equality between the two sides. However, mission schools did make contributions 
to modernizing the dilapidating education system in China at that time. “New schools” as advocated by many reformists 
were actually modeled on early mission schools. Mission schools were also the first to offer women’s education in 
modem China. The first girl’s school in China was established in 1844 by Aldersey, a British missionary (Hu, 1994). 
Mission schools started the first nurses’ training program in China. They also boasted some of the best universities and 
research areas in contemporary China. As we noted earlier, several of the best universities today like Tsinghua 
University and Tongji University started as mission colleges. 

2.2 Schools Run by Chinese Educators 

Private schools run by Chinese educators started later than mission schools. In the beginning, Chinese private schools 
were modeled after mission schools in order to pass on “Western studies” to new intellectuals and invigorate the nation 
with new knowledge. There are different understandings on the meaning of “Western studies”. At first, Chinese 
government felt the agony of its underdeveloped technology and machinery production. “Western studies” at this stage 
meant the learning of western machine production. Fujian Warship School and Shanghai Production Bureau were the 
two most important experiments in the hope of applying western technologies to industrial production. Later, Chinese 
intelligentsia came to realize that machinery alone could not save the nation. China had become weak mainly because of 
its malfunctioning political system. Western studies in this period meant the learning of western political systems. 
Finally, advocates of “Western studies” went further and asserted the defrciency of Chinese culture. They believed that 
within the context of old feudalistic culture, it was impossible to implement any real reforms. The failure of Wu Xu 
Legislature Reform proved that the new nationalistic political system could not possibly survive in the soil of old 
cultures. (Qiu, 1997). Through the development of understanding about Western studies, one constant theme was that 
education was consistently regarded as the most important way to rejuvenate the dilapidating nation. Both the 
government and individual citizens sponsored experiments to set up new schools. In 1862, Peking Tongwen Academy 
was founded by Yi Xin, the brother of the king. This was the first new style school in the history of China. This 
government school started as the language institute and later expanded with astrology and mathematics studies in 1 867. 



The prevalent thinking of education as omnipotent (Cai, 1984) combined with dissatisfaction with traditional learning 
institutions (Qiu, 1997), motivated great energy among open-minded gentries, merchants and returning overseas 
students to set up new private schools. Although some Chinese intellectuals began the experiment as early as 1878, this 
trend did not become obvious till 1904, when Guimao Education System brought fundamental and systematic changes 
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to traditional education structure. Therefore, 1904 can be viewed as the launch of modem private schools run by 
Chinese educators. Their development can be divided into four periods: from 1904 to 191 1, from 1912 to 1927, from 
1928 to 1936, and from 1937 to 1949. The division of stages roughly corresponds to that of mission schools. 

1904—191 1 . Zhang Huanglun founded Zhengmeng Academy in 1 878, which is the earliest modem private schools run 
by Chinese educators. However, as mentioned earlier, this type of school did not achieve much development before 
1904. The dissemination of evolutionary {gailiang, as opposed to more radical idea of social revolution) thinking since 
1898 and the Wuxu Legislature Reform illuminated Chinese people’s long shackled mind. More people were 
demanding education at the same time when more intellectuals were willing to provide education. Private schools began 
to gain in number. The growth in private education as well as the reformist atmosphere encouraged the Qing 
government to announce the first modem education system regulations in 1904. The implementation of this system, in 
turn, created a supportive environment for the new private schools to develop. The new regulations required local 
governments to found new schools in accordance with the 1904 law. An education ministry (Xue Bu) was established 
and the “eight-legged” essay examination was abolished shortly afterwards in 1905. Being aware of the shortage of 
financial resources and qualified teachers, the government encouraged individuals, most of them were old-fashioned 
intellectuals, to open schools to fill in the gap between what had been planned and the reality. The old gentry class 
responded to this appeal from the government passionately, considering themselves the backbone and leading class that 
should take the responsibility to educate its people. 



In this period, most of the private schools were primary schools. Most of them were concentrated in the coastal area, and 
they were labeled as “Chinese-Western studies academies” or “English schools” (Zhang, 1994). Although these schools 
attempted to imitate mission schools, most of them did not have any real breakthroughs and were still within the 
traditional school framework. In order to complete the new education system efficiently, learning promotion 
organizations were opened in every county to supervise the operation of new schools, both governmental and private. 

Although mission education began to focus more on higher education in this period, only a small number of secondary 
and tertiary schools were founded by Chinese educators. The Public University of China (1904) and Fudan University 
(1905) were the most important private universities founded by Chinese educators at this point. Nankai School, founded 
in 1907 by Chang Bolin, was the most famous secondary school at the time. 

According to Lin (1999), in 1906, there were 59 private secondary and primary schools, with a teaching staff of 606 and 
a student body of 3,855. However, private schools founded by Chinese educators suffered from an extreme shortage of 
funding. For example, the Public University of China, founded by returning students from Japan, had to rent dilapidated 
buildings as their classrooms. Many of the staffs were working voluntarily without salary. In an extreme case, in order 
to raise funds, Yao Honglie, one of the university administrators, committed suicide in the hope that “government 
administrators could use their political power, rich people could use their economic power, and intellectuals could use 
their knowledge to support the Public University of China together” (Hu, 1994). 

1^12 — 1927. With the overthrow of the Qing Dynasty, the Feudalism that had lasted for several thousand years came to 
an end. Various powers and influences began to compete in all fields of social life. In politics, starting from 1917, a 10- 
year war broke out between warlords across the nation before Guo Min Dang unified the country in 1928. In the field of 
ideas, republic thinking, democracy, communism and even Feudalist remnants were in conflict with each other, 
attempting to fill the vacuum left by the annihilation of Feudalism. The beginning of the Republican era created a 
relatively active and liberal environment for education. With the promotion of reforms by the new government, 
education developed quickly. The number of schools nationwide increased from 87,282 in 1912 to 129,739 in 1915, and 
the number of students increased by around 50% accordingly (Qiu, 1997). 

The active involvement of government in education development provided a strong support to the public education 
system. But at the same time, it also attempted to control private school ideology to corroborate its ruling position. The 
grip over education, fortunately, was not so strict as it might have been because of divisions created by the war between 
warlords that followed the founding of the Republican nation. Fighting for power reduced resources for education. 
Education fund was often appropriated for military use. Public schools were severely impacted by the war. By contrast, 
private schools, which did not rely on governmental funding, kept developing in a more stable fashion. In addition, 
because of the accumulating tension in Europe and later the First World War, the Western control over China was 
loosening. National industries in China thus got a precious chance to expand. The developing economy provided a 
strong source for educational investment. 



The first period of the National era (before 1922) saw a rapid growth in private higher education and professional 
schools of political science. “Regulations on Public and Private Vocational Colleges” and “Regulations on Private 
Higher Education” permitted individuals to open all kinds of new schools except for teacher training schools. The 
private universities outnumbered public universities in this period. Out of these private universities, 75% were 
vocational colleges, with most of them concentrating on political science (Hu, 1994). This reveals the passion for law 
and politics in the 1910s and 1920s. In contrast, primary and secondary education achieved greater success in the public 
sector. In 1912, there were only 54 private middle schools, accounting for 14.5 percent of the total number of middle 
schools in that year, and 12.8 percent of the total middle school population (Hu, 1994). 

A new education system was implemented in 1922. In the new system, requirements for setting up universities became 
less strict. The 4-year middle schools were changed into 6-year schools and were divided into junior high schools and 
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senior high schools, and they could be founded separately. As a result of these changes, the number of public 
universities exceeded that of private ones for the first time in 1922, with 10 public and 9 private. In 1927, 34 universities 
were public and 1 8 private. Many political science colleges were shut down as a result of their poor quality. In the 
secondary and primary education sector, however, a large number of junior high schools were established by individuals 
and the number of private middle schools reached 283 in 1925, or 41 .2 percent of the total number of middle schools of 
that year, enrolling about 40 percent of the whole middle school student population (Hu, 1994). 

1928 — 1926 . With the end of the chaotic struggling for power between warlords, the Guo Min Dang government unified 
the nation and created an uninterrupted period of stable social development from 1928 to 1936. The new government 
established a healthy and regulated education system. As early as 1927, the government announced its education policy 
based on the “Three People Principle”, expecting education to facilitate the creation and maintenance of national 
independence, the human rights equality, and the improvement of living conditions. Following the announcement of this 
policy, a series of regulations were announced. In December 1927, the government promulgated the “Regulations on the 
Registration of Private Universities and Professional Schools” to improve the quality of private higher education. To 
regularize the administration of private higher education, “Regulations on Private Universities” and “Regulations on the 
Private University Board Operation” were implemented in February of the next year. In 1929, “University 
Organizational Laws” and “Professional School Organizational Regulations” were announced to clarify such details as 
the minimum investment for starting a college, minimum operation investment and curricula (Hu, 1994). 



Besides the resolution of the government, other social factors also contributed to the relatively supportive environment 
for education. With less social turmoil, national industry achieved an annual growth rate of 8.7 percent between 1923 
and 1936 (Deng, 1997). The development of national economy resulted in greater resource availability for educational 
investment. At the same time, it also created a greater demand for skilled and educated labor. In addition, the prospering 
economy boosted the confidence and pride of the Chinese people. Educators started to seek new national identity by 
setting up new schools modeled after their own ideals of future China. For example, Cai Yuanpei, the most influential 
president of Peking University, advocated aesthetic education vehemently. He believed that aesthetics not only could 
substitute for religion, but was even better than religion in that “aesthetics is liberal, while religion is compelled; 
aesthetics is progressive while religion is conservative; aesthetic is popular while religion has boundaries” (Cai, 1984, 
pp. 501-2). With such an ideal, 3-year-old children were sent to the kindergarten to learn music, painting and literature, 
museums of arts were set up, theatres were built, professional arts schools were founded, and even pregnant women 
were sent to national infant education institutions that were set up in peaceful setting with fresh air (Qiu, 1997). 



Hu Shi and Cai Yuanpei also insisted on the independence of education. In his essay “On Independent Education”, Cai 
pointed out that: 



Education should develop individualism and commonality equally. What political parties attempt to do 
is to create a special kind of commonality and wipe out individualism. For example, they would 
encourage their people to love some nations, while hating the others; or try to use the culture of one 
ethnic group to absorb and dominate the culture of another group. This is the common practice of 
today’s party. It is extremely harmful if this practice was intertwined with education. What education 
aims at is the effect in the future, while political policy seeks instant change. ... the effect of education 
will not appear instantly. However, parties cannot hold their ruling position for long. Government 
changes within several years. If education is under the charge of parties, then when the government 
changes, education policy will change accordingly. No education effect can be achieved in this way. 

Therefore, education must be independent from political parties (Cai, 1984, p. 1 17). 

As mentioned above, the government in this period (as well as in all the other periods), attempted to control the 
education system. For example, In October 1925, the Beijing government ordered private colleges all over the country 
to shut down. This is an event that has not been explained to this day (Deng, 1997). The belief in educational 
independence was a counter force that protected and promoted the stable development of private education in the period. 
During the years of continuous wars and constantly shifting of powers, the appeal for education independence helped 
education survive. 

Another educator who significantly contributed to private education in China is Tao Xingzhi, also a Teachers College 
graduate and a disciple of Dewey. Combining pragmatism with the social conditions of China, he initiated village 
education, popular education, vocational education, wartime education and comprehensive education movements. He 
applied his ideals to practice and founded more than half a dozen schools. Xiaozhuang School, founded in 1927, was a 
village teacher training school, the first of its kind in contemporary educational history of China. With the belief that 
village teachers are the soul of countryside reform, Tao made his first experiment in the suburban area of Nanjing. 
Xiaozhuang School consisted of two parts: primary school teacher training and kindergarten teacher training. Guided by 
such principles as “life is education, and society is the school” and “integration of teaching, learning and practicing”, the 
school achieved great success quickly. Tao was invited by other regions to found similar teacher training schools in 
1928 and 1929 (Wang, 1982). His education ideas are still highly respected by the Chinese government today. 



o 
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At the beginning of the 1930s, with the purpose that science should be popularized among common citizens, Tao 
founded a children’s correspondence school in Shanghai (1932). In the October of the same year, he founded Private 
Shanhai Experimental School in Shanghai, which was a vocational school that combined general knowledge with skill 
training. In the next period of education development (1937 — 1949) when the Sino-Japanese war broke out, Tao was 
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concerned with the education in the battle field and founded Life Education Association and children’s school for the 
refugees. In 1939, Yucai School was established in Chongqing, the wartime capital. This school was an experiment to 
apply Tao’s ideal of comprehensive education. The school included six groups: music, drama, painting, literature, social 
studies and natural science studies. The school was so successful that its name is still used by many high schools in 
different cities across the country. 



With effort from the government as well as individual educators, private education scored impressive developments in 
this period. The percentage of private colleges increased from 27.6 percent in 1 925 to 49. 1 in 1 936. Correspondingly, 
private colleges enrolled 49.3% of all the college students in 1936, as compared with 35% in 1925 (Hu, 1994). Primary 
and secondary private education also developed on similar scales. In 1936, private primary schools and secondary 
schools accounted for 24.8% and 36.7% of the nationwide number of primary and secondary schools respectively. Most 
of the private schools concentrated along the costal areas and major cities. 

1937 — 1949 . War began in this period. Like mission schools, private schools run by Chinese educators also suffered 
severe damage and regular education could not be maintained. Many schools were demolished by bombing. The famous 
Nankai University was raided by Japanese invaders in 1937, the first year of war. Half of the private middle schools 
closed between 1937 and 1939 (Hu, 1994). Apart from the damage of school facilities, teachers and students were 
turned into refugees, creating an immeasurable loss to education. Public education suffered from the war as well. 
However, the government transferred some schools out of the occupied areas and was able to save some regular 
educational institutions. By contrast, privately invested schools did not have the ability to migrate to the inner lands. 



Around the end of Sino-Japanese war in 1 945, the government began to offer some support to private schools. Yet in 
return, many schools were changed from private to public. Examples included Nankai University and Fudan University, 
two of the earliest private higher education institutions. Not only the number of private schools decrease, but also their 
quality deteriorated with some of the higher quality schools becoming public. Although private education gained some 
respite after WWII, the civil war that followed between Guo Ming Dang and the Communist Party undermined the 
ability of private schools to survive. 

3. Post 1949 era 

The shifting focus of national development in China after 1949 reveals the struggling for control between two fractions 
inside the Chinese Communist Party; the radicals (as represented by Mao Zedong) and the moderates (as represented by 
Liu Shaoqi and Deng Xiaoping) (Tsang, 2000a). The radicals put emphasis on ideological struggle, while the moderates 
focus on economic and material improvement. The overall policy changing decides the shifts in education policy 
(Tsang, 2000a). Although education development in the post 1949 era can be divided into more periods (Tsang, 2000a), 
for the purpose of analyzing private education, two major stages are suffice to see the change: from 1949-76, when 
private education was first severely suppressed and then totally eradicated; and from 1976 to the present, when private 
education revived and is still gaining importance in the overall education system. 

3.1 1949 — 1976. The Termination of Private Education 

After the founding of the People’s Republic of China, all of the private sectors of social life were transformed into 
public sectors. This is consistent with the nature of the Chinese Communist Party as a communist party. Equality was 
regarded as the most important principle, and individual goals were suppressed by collective goals. Private industries 
and schools were regarded as the manifestation of Western capitalism, which was in the direct opposition to 
communism. Within such a radical ideological framework, schools began to be folded into the public sector in 1951. 
Mission schools were also among those on the conversion list. Furen University was the first mission university that was 
transformed. It became part of Beijing Normal University, bringing with it a great collection of precious books. By the 
end of 1952, private education had evaporated in China. 

Private education did not reemerge until 1978. Between 1952 and 1978, there was only one type of school that existed in 
large numbers in the vast rural area: people-run schools. People-run schools are schools “sponsored and managed by a 
community of people or a collective organization, and funded by resources from the community or collective 
organization, and from a variety of sources” (Tsang, 2000b, p. 4). 

This type of school came into being in response to the huge gap between the supply and demand for public education in 
the poverty-stricken rural areas, where around 80 percent of the total population lived. Parents usually could not pay any 
tuition, and teachers received no regular salaries. Instead, villagers provided food and room for the teachers and helped 
in such activities as school building, and teachers had to move from home to home for accommodation. The government 
offered almost no financial support for these schools, but at the same time the party maintained a tight control. People- 
run schools had neither administrative autonomy nor academic freedom (Deng, 1997). For this reason, though privately 
funded, people-run schools are not categorized as private schools (Deng, 1997; Tsang, 2000b). 

3.2 1976 — Today. The resurgence of private education in PRC in post-Mao era 

After the chaotic “10-year Cultural Revolution” ended in 1976, the less radical faction in the Chinese Communist Party 
rose to power under the leadership of Deng Xiaoping. The Third Central Meeting of the Eleventh Party Conference 
decided on a new package of national development policies, which is known as “reform and open” strategy. In the 



process of reform, the Party further liberated its thinking and started the creation of a “socialist market economy”. 
Education’s role in national development is no longer ideological; instead education has the important function of 
meeting the skill requirements of a developing socialist market economy and is portrayed as the strategic foundation for 
national development (Tsang, 1996, p. 54). With excess demand from the society, and with the permission of private 
business in a socialist country, private education reappeared in China. 

Definition of private schools. Different definitions of private education will result in different categorization of public 
and private sectors. Usually there are two standards: Is it privately funded and is it privately managed. According to the 
Regulation on Education Run by Social Forces, instituted since July 31 , 1997, private schools (or schools run by social 
forces) refer to those run by "businesses and governmental organizations, social groups and other social organizations 
and individuals, using non-government educational financial resources, to provide schooling and other forms of 
education to the society." (Lin, 1999). 

Context of private school resurgence. The potential education market is created by excessive social demand as 
compared with limited governmental supply of education, together with the consumer’s willingness and capability to 
pay. Usually private education comes into play when there is either an absolute shortage of education such that not 
everybody has access to schooling, or a demand for education alternatives that the existing system cannot satisfy 
(James, 1 995). According to Lin (1999), in developing countries, the rationale for the existence of private education 
tends to be very different from that in more developed countries. Camoy and Samoff (1990) see private education in 
developing countries as an inescapable solution to the rising demand for education, particularly at the secondary level. 
Parents who send their children to private schools are not necessarily exercising a constitutional right of choice, but 
rather solving personal problems or using a system that increases their children's chance for social mobility. The 
absolute shortage of education supply seems to be the main reason for the existence of private education. 

This is also true in the case of China. Shortage of funding proved to be devastating to China’s public school system 
(Deng, 1997). China’s educational budget between 1950 and 1985 rarely exceeded 3 percent of its GNP and was overall 
0.7 percent less than the international average (Ho and Mao, 1992). The under-invested education system, coinciding 
with a huge and increasing population, makes the situation even worse. Examinations were widely adopted as a 
mechanism of competition for the limited education resources. Primary school students competed for “key” junior high 
schools, and junior high students compete for “key” senior high schools, with these sought-after schools providing 
nearly sure access to higher education. At the end of high school years comes the notoriously competitive college 
entrance examination. At each of these stages, a large number of students are denied the chance for further education. 
When private schools came back in the early 80s, their targetted market was these “failed” students, offering them the 
kind of training needed to compete for college enrollment for a second time. 

The demand for such second-chance schools outside the public education system was made even larger as a result of the 
Cultural Revolution (1966 — 1976). During those years, youths were denied access to regular higher education and were 
sent to the countryside to receive “re-education” by laboring on the farms. Once the normal higher education system 
was restored in 1978, the huge number of college candidates that had accumulated in the previous 10 years began to 
take the entrance examination at the same time. Such severe congestion inevitably left a large number of failed students 
who created a demand for further training. 

The market of “second-chance” students still exists today. Public compulsory education is not absolutely free. Though it 
charges no tuition, fees are nonetheless collected under various names. In underdeveloped areas, children have to travel 
across hills, rivers and vast farming fields to get to their schools. When all the fees, traveling costs and forgone earnings 
are considered, public education is so expensive that impoverished families in some rural areas cannot afford it. Private 
schools come up catering to the education demand of these families. In big cities and the coastal region, private schools 
offered a second chance to those academically unqualified students from rich families that could afford the private 
tuition. For those who failed to get admission to public universities, private universities, many of them correspondence 
colleges, rose to meet the demand of adults. 

Starting in 1992, when the first elite high school was founded, demand generated by supply shortage was no longer the 
only market for private investment. With the booming economy, people began to seek more school alternatives beyond 
what the public system offers. Economic growth brought a sharp rise in family incomes, giving people the financial 
foundation of choice. At the same time, the economy has been shifting from a central-planning system to a socialist 
market economy. This has increased the demand for skilled labor force, for which private schools have responded 
quickly. 

With economic development being regarded as the primary task of national effort, social values have changed. The ideal 
of egalitarianism has been abandoned for the sake of faster growth. Between being poor but equal, and being rich but 
unequal, the government has chosen the latter. Wealth has become the common goal of the society. Although political 
connections prove to be the most effective method of wealth accumulation, education is the path to higher economic 
gains for the majority of common citizens who can afford it. To ensure a “brighter” future, parents want to choose the 
“best” education for their children. The great demand for certain skills in the labor market is reflected in the great 
demand of training in those skills. For example, for most families, key middle schools would be their ideal choice. But 
since these schools are not accessible to most of them, and yet they are not satisfied with common high schools, private 
schools with better facilities and more flexible curricula become their second best choice. For some families where both 
parents are involved in business with little time to care for their children, an alternative schooling that can take care of 
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both the study and life of students is attractive to them. In recent years, it has become more and more popular for 
families living in the central and western regions to send their children to big cities in the coastal area so that they will 
be more adaptable to the metropolitan and modem environment in the future and get more job opportunities. It is 
impossible for these students to get enrolled in the local key schools of big cities. But money can buy them places in 
private schools. 

Other factors helped enlarge the market demand for school alternatives. For instance, the one-child policy, a method of 
population control, created a large number of nuclear families with only a single child to educate. This policy came into 
effect around 1976. These children began to compete for secondary education in the 90s. No parent would like to see 
failure in the training of the only child of the family. In addition, the parents of these children suffered from the Cultural 
Revolution, and most of them were denied the chance to receive even secondary education. They would never want this 
to occur to their children. As a result, they have made efforts to seek the best education available to their children. Some 
parents from cities were sent to remote areas in the revolutionary years. Many were unable to return to their hometown, 
even after the end of the revolution. Their hope lies with their children: If their children can get admitted by universities 
in cities (especially Shanghai and Beijing), current policy allows the parents’ hukou (Note 3) to return to the city with 
their children. Private schools in eastern cities also give these parents a good choice to realize their dreams. 



Another factor, though not quite clear in its nature and scale, has contributed to the market for elite private schools. 
Economic growth has brought more illegitimate children, bom to the mistress of rich businessmen. Unofficial 
information suggests that there is a population of nearly 100,000 mistresses in the Zhujiang area alone. Unfortunately, 
these children are not permitted in regular schools. At the same time, many of the rich fathers are more than willing to 
pay for the education of their children. This generates a market demand for private education, especially in the special 
economic zones in South China. 



Such a great and heterogeneous demand described above is not sufficient by itself to bring private education back into 
existence. Privatization in education follows the privatization of economic production in the country (Tsang, 2000b), 
which is part of the ‘‘socialist market economy” policy. Besides, with the increasing GDP per capita, families now have 
the capability to consume private education. The overall policy and household income make people’s education demand 
realizable. 



Regulatory environment. Alongside the burst of demand for private education, the government created a favorable 
policy environment for the growth of private schools. As early as 1985, some documents on “structural reform of 
China’s educational system” allowed university departments to find part of their resources through engaging in business 
activities or through enrolling a certain number of students outside the admission quota set by the government (Lin, 
1999). This was the beginning of education decentralization. A series of regulations and laws concerning non-state 
schools were issued afterwards. The key ones that are regulating today’s market are: the 1993 Provisional Regulations 
on the Non-State Higher Educational Sectors, the 1995 Provisional Regulations on Education Institutions Jointly 
Sponsored with International Institutions, the 1995 Education Law, the 1996 Vocational Education Law, the 1997 
Regulations for the Non-State Education Sector, and the 1998 Higher Education Law (LaRocque & Jacobsen, 2000). 

These regulations provide a supportive stance for private education in the PRC. The Law stipulates that private 
education institutions should be non-profit organizations, and the surplus shall not be distributed among investors. 

Under the current laws, private schools are not entitled to public education funds. In fact, from 1985, the government 
began to supplement state funding of education from other sources. Its policies are consistently seeking more money 
from student families rather than increasing the governmental investment in education. More and more of the financial 
burdens of education are being transferred from the state to individuals. Even public education is no longer free. 

Students enrolled in regular public universities began to pay tuition from 1994. 

The risk to private investment in education is the informality and lack of transparency of the regulatory environment in 
China (LaRocque & Jacobsen, 2000). Enterprising individuals were ready to assume financial risks as well as the 
political risk of being criticized or condemned by the government should it reverse its policy (Kwong, 1997). But such a 
loose regulatory environment also generates some flexibility in the implementation of regulations. Some private schools 
successfully acquired support from the local government, and non-profit private schools are actually collecting surplus 
through various ways. Some investors started businesses affiliated with their schools. Profits gained from education can 
thus be transferred to company surplus and become legal profits. Policy makers are attempting to come up with different 
interpretations of school’s “non-profit” status. In fact, the legislation on private education is one of the hottest topics 
among Chinese educators. Debates mainly concentrate on for-profit or non-profit status, ownership, and even the legal 
name for private education: Should it be “people-run schools”, or “schools run by social forces”, or just private schools 
(Wang, 2001). 

Development of private schools. The development of private schools in the post-Mao era is divided into three stages 
(Lin, 1999): 1978 — 1987 is the first period, when most private schools were training institutions and night schools 
targeting second chance students. The second period saw the appearance of private regular schools. And the third period 
started from 1992, when the first elite school appeared in China, signaling the advent of rapid growth of various types of 
private schools to the present day. 

In 1994, private schools constituted less than 4 percent of the country’s schools (Kwong, 1997). The distribution of 
private schools is not balanced. In Wenzhou city of Anhui province, private secondary schools made up 51% of all 
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secondary schools in 1996 (Zhang, 1996). Private schools are developing quickly in some economically underdeveloped 
region like Yunnan Province as well as in the affluent cities. According to the survey done by LaRocque and Jacobsen 
(2000), in 1998, there were nearly 42,000 private education institutions in China. Of these, 85% were at the pre-school 
level, 1 1% were at the elementary and secondary levels and 3.5% were at the tertiary level. Excluding the tertiary sector, 
these private institutions enrolled 6.5 million students in 1998. In 1997, there were over 1200 private universities 
existing in the country (Lin, 1999), with 37 of them having the right to confer degrees. 

Dijferent types of private schools. Private schools are often classified into three major types: urban elite primary and 
secondary, ordinary private schools, and private universities (Lin, 1994). Urban elite schools attract the most attention 
from the society because of their extremely high tuitions and construction fees charged to students. They are usually 
boarding schools that have substantial resources. Teachers receive salaries than public school teachers and schools often 
have additional requirements beyond the regular curriculum requested by the government. Students are admitted on 
family’s capacity and willingness to pay instead of academic achievements. But to ensure a certain level of quality, they 
sometimes provide scholarships to attract top students from key schools and persuade poorly performing students to 
transfer to vocational schools before the college entrance examination (Lin, 1999). In a sense, elite schools provide a 
chance for some rich students to “buy” the right of leaning. 



Ordinary private school includes rural private schools, single-sex schools and art schools (Lin, 1999). They are 
affordable by the average families and are usually secondary schools. Charging much lower tuition, these schools are 
less profit-oriented than elite schools. In fact, many rural private schools were founded in response to the high and 
unaffordable charges of public schools. To some extent, they are similar to the people-run schools between 1952 and 
1978. These schools typically have meager finances and resources. Most of the teachers are part-time or retired teachers 
seeking extra income besides their regular salaries (Lin 1999). Some schools in this category also offer training in 
foreign languages, computer skills, or examination preparation classes. 

Although it is claimed that private universities enrolled one quarter of the total college student population in 1995, a 
great proportion are correspondence students. Their targeted market is adults instead of high school graduates. Usually 
private universities offer limited professional training in a narrow range of subjects that are popular in the job market. 
There are three types of private colleges: The first type has the right to confer degrees independently. The second type 
can issue joint degrees with other regular institutions, and the third type only provides training for students to take the 
Adult Self-study College Examinations, which lead to a college diploma equivalent. The last type of private tertiary 
training had 1,080 schools in 1998 (LaRocque & Jacobsen, 2000). Private colleges are usually affordable to common 
families. To survive on limited financial resources, they mostly employ part-time teachers or senior students from 
famous public universities. 



Investors in private schools. Investors in education range from business entrepreneurs to retired teachers, government 
officials, overseas Chinese, and public schools. They invest in education for different reasons. Many of them are driven 
by economic profit. Some are dissatisfied with the existing education system and thus carry out their own experimental 
approaches, just as Tao Xingzhi did in the 1920s through the 1940s. Some business people set up schools just to obtain 
profitable land and tax benefits for schools. The most interesting and sometimes ironic phenomenon is that of public 
schools investing in private education. Starting from 1985, universities have been allowed to raise financial resources 
outside of state funding by operating a business or admitting extra students. With the existing facilities and teachers at 
hand, it is so easy to set up a short training session so that adults outside the campus can have partial access to higher 
education, while teachers can get some extra money to improve their lives. Setting up a night school appears to be a 
more legitimate and fair way of generating revenue for the public schools than admitting extra students who actually use 
money to compensate for their academic inadequacy. In addition, starting a private school on the basis of an existing 
public school is much easier and safer than starting a new one from scratch. 



Operation. Lin (1999) identified four sources of funds for private schools: state funds, fees charged to parents, income 
from operating school businesses, and income from offering extra classes. Other sources also include equity and short- 
term bank loans (LaRocque & Jacobsen, 2000). Among these, a significant amount comes from fees charged to parents. 
For elite high schools, this category consists of tuition, construction fees, education savings and other fees like 
transportation and uniform. Education saving fund is way higher than normal tuition (around 10,000 to 30,000 USD). 
Schools promise to return the fund in the original amount at the time when the students have finished study. This form 
of deposit plays two important roles: The huge amount of money can be used as an interest-free loan from parents, so 
that schools can afford expensive initial construction. Second, during the years of study, schools can benefit from the 
interest on deposits or returns to other investments that schools make. In a rotating fashion, the deposit withdrawn by 
graduated students is replaced by the deposit from in-coming students. Though the capital flows all the time, its stock is 
constant. The deposit becomes fixed school property. 



Private schools have more independence in administration. Under the general guidance of the government, they can 
employ their own teachers and administrative staff quickly and make changes to the curriculum without the approval 
from the local government. Teacher’s salary is used as an incentive to better performance. It seems that in this way, the 
problem of stagnant curriculum that tends to stifle creativity of students in the public schools can be solved by the 
flexibility of private schools. Such ceremonies as flag-raising every morning in the public schools could be overlooked 
or quietly sacrificed for academic excellence (Deng, 1997). However, few studies have been done to compare the 
efficiency between public schools and private ones. It would be premature to claim that the curriculum of private 
schools is better than that used by public schools just because they have added several more computer classes or piano 
classes. And since love of one’s nation and people is one of the essential tasks of education, the flag-raising ceremony 
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might be a good method of cultivating pride in one’s own country. But it is doubtful whether the couple of minutes 
saved from the ceremony really can improve academic excellence much. Other practices of elite private schools, such as 
the boarding requirement and typical weekday timetable (Lin, 1999, p. 64) do not seem to differ a lot from those of key 
high schools. Many of the “innovations” seem to be designed for marketing purposes rather than educational advantage. 
In addition, in some cases principals do not have the right to decide on financial issues. Allocation of resources is under 
the control of investors who may have no experience in the education field. 

Concerns about private schools. The personal accounts nationwide accumulated a total sum of deposit exceeding 2 
trillion RMB (or more than 250 billion USD) as of 1994 (Deng, 1997). The government is trying to direct the citizens to 
consume more education out of their own purse. People stay longer in schools. Public universities and graduate schools 
are expanding their student population quickly, in some cases by 20% per year. It seems that the government is pressing 
to expand the current educational capacity using private resources. One concern about private education in such an 
environment is its over-heated growth, which, in turn, may possibly result in improper competition such as the using of 
dishonest advertising. Another concern is that the government appears to care more about earning money by using its 
education institutions than educational quality. A lot of residence halls and classrooms are being built around 
universities, while the number of regular teaching faculty remains unchanged. Graduate students are compelled to teach 
“voluntarily” some big classes completely by themselves. 

A more serious concern is the inequality problem. Deng (1997, p. 136) pointed out that “private schools that prospered 
on the growing gulf between the rich and poor only magnified the problems that were besetting Chinese society in the 
1990s”. Compared with the past, rich students have one more way to get education: Buy it. However, even if there were 
no private schools, the “enrolling extra-quota students for financial resources” policy has already give privilege to the 
economically advantaged group. Besides, as mentioned above, rural ordinary private schools actually create chances of 
education for the poor who cannot afford the public education. The relation between education equality and the 
development of private education requires more detailed study. Actually the current private education development 
policy is confusing in that on the one hand, policy makers are making great efforts to re-interpret the “non-profit 
education” regulation so that entrepreneurs can distribute profit legally (because researchers believe profit is the 
incentive), while on the other hand, educators also realize the importance of promoting private education in poor areas 
to meet the excess demand there, forgetting little profit can be further wrenched from those families (Yang, 2001). 

There are other worries about private education. For example, private schools usually offer classes on “hot” skills. Now 
it is not rare for some kindergartens to teach children English before they can speak Chinese fluently. Computer classes 
are the emphasis of many private high schools, while such subjects like math is ignored (Lin, 1999). This is detrimental 
to the establishment of a solid knowledge foundation for students. Besides, there is no systematic evaluation mechanism 
for the outcomes of education. Little information on student performance in private schools is available. The most recent 
conference on private education policy held in Hangzhou this summer was abundant with articles of casual “thoughts” 
instead of serious assessments. 

4. Conclusion 

The available literature shows a lack of systematic experiment and evaluation in the study of private education policy. 
The description of private school development is limited to the amount of investment, the number of schools, the 
students enrolled, sources of fund, the fees charged, and so on. With nearly 10 years of rapid development since 1992, 
the outcomes of private education can be and should be measured to determine its contributions and social 
consequences. With the continuing expansion of public education, private education may face serious challenges in the 
future. 



As demonstrated by the private education history, China has started its non-govemmental schooling since Confucius 
began to provide education to people outside the government and the ruling class. The current private education differs 
from the Chinese tradition in that today’s education is unprecedentedly commercialized and market-oriented. Education 
history tells people that education is a career that requires devotion from the teachers, that education is a science whose 
value lies with truth, and that education is an art, whose vigor comes with innovation. Resource diversification and 
expansion needs to be balanced with quality. 

Notes 

1. The 2010 education goals include: Reduce the young people illiteracy rate to less than 1% and raise the total adult 
literacy rate to 90%; The junior high school gross enrollment rate reaches 95% so that 95% of students can complete the 
9-year compulsory education; The gross enrollment rates of senior high schools and colleges will increase to 50% and 
1 1% respectively (State Education Commission of PRC, 1996). 



2. The eight-legged essay has played a famous (and sometimes infamous) role in Chinese literature. It began as an 
attempt to give an ordered form to the essay and eventually became a standard part of the Civil Service examination. As 
time went on, it began to be no more than a rigid and lifeless exercise that all examinees were expected to perform. 
However, some of the earlier essays successfully conveyed real messages briefly and tellingly within the highly 
regulated eight-legged format. (See http://www.wlu.edu/~hhill/baguwen.html/ for more information) 
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3. Hukou is like citizenship. But citizenship is used to differentiate country identity and control population movement 
across countries. Hukou is used to differentiate city, county and town identity and control free migration across different 
areas within a country. 
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Abstract 

Since the 1983 report, A Nation at Risky the performance of public schools has been increasingly 
scrutinized, and a variety of reforms designed to increase student achievement enacted. Among 
the reforms discussed, much attention has focused on increasing choice and competition in 
education. While the effectiveness of market oriented reforms have been widely debated, little 
research has been completed that examines policy maker attitudes toward market reform of 
education. This study used a researcher designed survey to examine policy maker attitudes 
toward education and education reform in general, as well as the issue of vouchers more 
specifically. Findings suggest that policy makers generally accept the market arguments used by 
voucher supporters, but are also sympathetic to equity concerns and funding issues raised by 
voucher opponents. Additionally, while more policy makers responding to this survey supported 
some type of voucher program than opposed vouchers, when viewed in the broader context of 
reform options, vouchers did not rate highly. 



Introduction 

On June 27, 2002, the U. S. Supreme Court ruled that the use of vouchers to pay tuition at private K-12 schools in 
Cleveland, Ohio, did not violate the establishment clause of the U.S. Constitution (Zelman v. Simmons-Harris, 2002). 
This ruling energized the pro-voucher movement, and resulted in plans by state iegisialurs across the country to 
introduce new voucher legislation (Toppo, 2002). While there is debate as to how far reaching this decision may 
ultimately be (some individuals have compared the decision to the 1954 Brown v. Topeka Board of Education 
decision), observers generally agree that the policy focus, both legislative and judicial, regarding the use of vouchers 
for K-12 tuition, will now turn to the states (Gehring, 2002; Toppo, 2002). In fact, less than two months after the 
Zelman decision. Justice Kevin P. Davey of the Leon County Circuit Court, struck down Florida's state-wide voucher 
program as violating that state's constitutional prohibition on state aid to religious institutions {Holmes v. Bush, 2002). 
The Institute for Justice has also filed law suits in two states (Maine and Washington), seeking to have state-level 
prohibitions against the use of public funds at religious schools overturned (Institute for Justice, 2002). 

As voucher policy is drafted, introduced and debated, the views and beliefs of state legislators will play a critical role. 
To help determine just how policy makers react to vouchers and school choice issues, a survey of state legislators was 
conducted in November, 2000. The results of this research are detailed in the accompanying article. 

398 




Despite interest from policy makers and advocacy groups, little research has been completed that would lend to a 
clearer understanding of the effectiveness of arguments used to promote or inhibit voucher plans and how policy 
makers respond to such arguments. The study conducted here attempts to examine policy maker reaction to these 
arguments, using the following questions: 



• What role do policy makers feel vouchers play in the larger context of reform? 

• How do policy makers react to specific policy arguments about vouchers? 

• How does the educational philosophy of individual policy makers relate to their attitude regarding vouchers? 

• How do the demographic traits of policy makers relate to an their philosophy of education? 

• How do the demographic traits of policy makers relate to their views on vouchers? 



This study specifically examined efforts to privatize education services through the use of vouchers by looking at three 
states where vouchers were enacted (Ohio, Wisconsin, and Florida), as well as three states where vouchers have not 
been enacted, but where serious attempts have been made to establish such programs (Michigan, New Mexico, and 
Pennsylvania). By examining policy maker reaction to arguments designed to promote or prevent voucher programs in 
states where vouchers were established or under serious consideration, this research sought to help both proponents 
and opponents focus on the concerns of policy makers in education reform. 

Background 

The education market movement is a compilation of at least three different efforts to expand the role of private 
providers in public education. These efforts include contracting out support and curricular services (such as food 
service, student transportation, and curriculum programs); contracting with a private company for management of an 
entire school or district (such as through Edison schools); and privatization of school governance (charter schools, tax- 
credits, and vouchers). While the term privatization may encompass all of these movements, as it is used here, 
privatization, vouchers, or market reform will generally refer to the privatization of governance through the transfer of 
public funding from public schools to the private sector. 

Method 

It is not until recently that the use of vouchers as a mechanism for creating a free-market system has been seriously 
considered; and not until the 1990s that a useful model of a voucher system established. The social and political 
contexts under which voucher programs were established then are relatively recent. This recent history presented some 
unique methodological opportunities (the chance to survey actual participants in the policy making process) and 
difficulties (no conclusive outcome or evaluative measures, continued doubt as to the constitutionality [at the state 
level] of such programs). For a detailed description of the methodology used in this research, see Appendix I. 

In general, a quantitative methodology that allowed for wide spread application and comparative analysis, was sought, 
and consequently, a survey of legislators was chosen as the best option for answering research questions across a wide 
geographic range and large body of potential data sources. The focal points of this study (Florida, Michigan, New 
Mexico, Ohio, Pennsylvania, and Wisconsin) were chosen using both critical case sampling and politically important 
case sampling (Martella, Nelson, & Marchand-Martella, 1999). 



The survey was constructed using pro- and anti- voucher arguments uncovered during a review of voucher literature. 
During the validation process, the survey was amended to include questions related to broader issues, including policy 
maker views on the purpose(s) of education, as well as the potential effectiveness of a variety of current reform 
proposals. Demographic data, including race/ethnicity, gender, age, and religious affiliation, was gathered to test for 
any specific response patterns. On October 18, 2000, the survey was mailed to 936 state legislators in Florida, 
Michigan, New Mexico, Ohio, Pennsylvania, and Wisconsin. Of the 936 surveys mailed, 89 were returned, for an 
overall return rate of 9.5% (see Table Al, Appendix 1 for a detailed discussion of the return rate and Appendix 2 for a 
discussion of methodological issues). 

General Overview of Findings 

Just under half of the policy makers surveyed (48%) supported some sort of voucher reform, however, when asked to 
evaluate vouchers in isolation, and when examined in relationship to other reform options, their support weakened, 
with vouchers ranking last among eleven other reform options. Policy makers also viewed the broader concept of 
school choice more negatively, with three of the four lowest ranked reform options on this survey related to school 
choice (teacher preparation and professional development, early childhood initiatives, phonics based reading programs, 
and greater use of technology all rated higher as reform strategies). 



While choice was not favored in comparison to other reforms, policy makers still generally seemed to accept pro- 
market arguments for voucher programs, as well as statements related to allowing consumers to regain control of 
educational systems. While respondents seemed to accept at face value statements that competition will improve 
services, they did not support statements related to the potentially negative impact of market forces. 
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Even though there was broad agreement on the benefits of a pro-market approach, there were concerns expressed 
relative to resource allocation. Respondents did not support the idea that vouchers would help equalize funding, or 
provide new schools, increased investment, or improved cost controls. In fact, they felt that voucher programs might 
result in reduced services to special education students and increased competition for the "best" students. Respondents 
were also concerned that private schools would raise tuition and fees, ultimately limiting accesses to the schools by 
poorer families. 

One of the most common warnings of voucher opponents is that voucher programs may result in a separation of 
students by race. Three items were related to this concern were included in the survey. Voucher advocates supported 
two of the more positively worded items (that vouchers would force schools to focus on customers seeking specific 
academic, social, or religious programs, and that such programs would result in schools with specific religious 
affiliations or racial/ethnic compositions), while voucher opponents were more likely to support the more negatively 
worded suggestion that voucher programs would increase segregation by race, religion, or income. 

While some policy makers expressed concerns that vouchers would result in private schools loosing independence, 
others worried that there would be a jump in the number of low quality schools, as entrepreneurs sought access to 
newly available public funding (this dichotomy between independence and oversight has played out in all of the 
programs enacted so far, which currently include either no, or very weak, evaluation and oversight components). 

When examining education more generally, respondents agreed that the overriding purpose of education is to ensure 
academic excellence in students. Respondents also supported the statement that education should ensure that students 
are prepared to meet the needs of businesses and employers, however voucher advocates were much more likely to 
want schools to instill strong moral character in students, while voucher opponents wanted schools to create good 
citizens. There was also a general reluctance to rank social purposes-including the suggestion that education should be 
used to promote social mobility or diversity-strongly. Women as a group tended to rank the creation of good citizens 
more strongly than men, but that emphasis came at the expense of business. Respondents who were minority group 
members were slightly more likely to rank social mobility higher than Caucasian respondents. For both women and 
minorities, however, the degree of difference between their opinions and that of the majority was not statistically 
significant. 

Item Analysis ^ 

Demographic Data 

The typical respondent to this survey was a white protestant male, however, the diversity represented by the 
respondents is similar to that of legislators nationwide and across the study states (see Figures 1 & 2), Approximately 
79% of respondents were male, and 1 9% female, with 2% not answering (this figure remains constant and so will not 
be repeated). Approximately 80% of respondents were white, with 6% identifying themselves as black, 1 1% Hispanic, 
and 1% Native American, The age range of respondents is shown in Figure 3, The large majority of respondents were 
actively religious (87%) and Protestant (65%), 26 percent were Catholic, and 2% were Jewish, Most respondents had 
completed a Bachelors degree and many had gone on to post-secondary education (BA, 46%; Professional degree, 
15%; MA, 14%; Ph,D,, 7%), A slight majority of respondents were Republican (53%) while Democrats made up 45% 
of the responders, A large majority had also served on the education committee (66%), Most respondents attended 
public K-12 schools (72%), and none attended private non-parochial schools at the K-12 level. Most of the policy 
makers also attended a public college (59%), while those who went to a private colleges and universities were spilt 
between religious institutions and nonreligious. While 72% of respondents who had children sent them to public 
schools, the 28% of respondents who sent their children to private schools chose parochial schools over non-parochial 
ones (83% tol 7%), While education service was generally low (ranging from 0 - 6% in most categories), of those 
individuals who had worked in education, 27% said they had worked as a public school teacher, 22% had volunteered 
in a public school, and 9% had volunteered at a private school. 
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Abstract 

Our analysis begins with an examination of the state of Georgia’s rationale for the decision 
regarding social promotion that was based on the perceived views that teachers have on the issue. 

Research suggests, however, that teachers hold contradictory opinions concerning the use of 
standardized tests for high stakes decisions, such as promotion, and are not aware of the 
consequences most children suffer when they fail a grade. Following a discussion that challenges 
the claims of success in Chicago, Baltimore, and Texas, we explore the viability of choosing 
litigation as a strategy to stop the use of high stakes tests given the adverse impact they have on 
protected minorities. From a study of the thirty-nine poorest counties in rural Georgia, the 
relationships between poverty, race and the Georgia Criterion Referenced Competency Test 
Results suggest that these tests do have an enormously disparate impact on impoverished African 
American children. Because chances for educational attainment will be severely limited by this 
test, most African American children will be discouraged from achieving a high school diploma. 

As a way to put a face on the data, a case study of a young girl who would probably fail her 
grade in school if the law was enforced is presented followed by recommendations that argue for 
changes in education policy and teaching. Rather than mandate a ban on social promotion, the 
state of Georgia should pursue improvement of socio-economic conditions, education policy and 
pedagogy. 

For many, it is simple common sense to fail a child who does not pass the academic requirements for promotion to the 
next grade. For others, making children accountable for their academic performance is part of an overall strategy to 
raise the quality of education. Responding to the public and political call for educational reform and accountability, 
Georgia and Texas, Baltimore and Chicago along with many other school districts have decided to end the practice of 
social promotion in hopes of improving the quality of education (Eisner, 2000) 

Defining social promotion is somewhat difficult because the meaning behind the practice has become so infused with 
professional, political and academic agendas that it seems impossible to reach common ground. On one side are those 
who deride the practice as promoting a student from one grade to the next regardless of academic achievement; they 
claim that this is a policy that short-changes the child, teacher, school and society (DiMaria, 1999). On the other side 
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of the debate are those who point to the abundance of research that overwhelmingly suggests that keeping a child with 
his or her peer group is the best insurance for high school graduation (Darling-Hammond, 1998). Countering this 
plethora of research is the assumption that a high school diploma has little value if the student is simply passed along 
and given a degree without meeting specific criteria for graduation (Eisner, 2000). As evidence of the rift in the 
stakeholders' views, the two major teacher organizations cannot agree on the subject. The more conservative 
American Federation of Teachers (AFT) shudders at the thought that school districts ignore policies and laws that ban 
the practice of social promotion by stating that this rampant disregard creates a huge class of ill-prepared and 
unmotivated students (AFT, 2001). While conceding that social promotion without intervention is deleterious to a 
learner, the National Education Association (NEA) is on record as stating that retaining students is even more 
pernicious (NEA, 2001). 

Examining Georgia’s Decision to End Social Promotion 

As a top priority for his education reform agenda, Governor Roy Barnes urged the legislature to end social promotion 
during the State of the State address in February 2001 . Governor Barnes proclaimed, "the time has come to end social 
promotion in our schools" (Barnes, 2001). Reasoning that social promotion is unfair to teachers, Governor Barnes 
charged the legislature with passing a bill that would require every student to pass an exit examination before being 
promoted to the next grade (Barnes, 2001). Using the rationale that schools and teachers are held accountable for 
competency, the Governor insisted that no student be promoted to the next grade level until proficiency in the subject 
matter has been assessed with a criterion based standardized test (Barnes, 2001). In Governor Barnes’ words. 



Now, nobody wants to have to hold a child back in school. It is difficult for them to be separated 
from their peers. But if some children are still behind even after we have taken every step available to 
give them extra help— after school programs, alternative programs, special reading programs and so 
on— we owe it to them to make this difficult choice... . But mostly, we should do it in fairness to those 
students who are passing through our system today without learning what they need to know. By 
promoting a child who is not really ready, we say, 'It's OK if you don’t learn.' Well, I say, it is not 
okay (Barnes, 2001). 

On March 21, 2001, the legislature of Georgia complied with the wishes of the Governor by passing into law a bill 
which mandated that students in grades third, fifth, and eighth must pass a standardized examination to move up to the 
next grade, beginning with third graders in 2004. Children who enter third grade in 2003 would be required to pass a 
state reading test, while those matriculating the fifth grade in 2004 and eighth grade in 2005 would be required to pass 
both a state reading and mathematics exit examination (State Board of Education [SBOE], 2001). 

A second chance to pass the test is allowed if the child fails to pass the test on the first attempt. If the child should fail 
a second time, a grade placement committee is convened "composed of the principal or the principal's designee, the 
student’s parent or guardian, and the teacher of the subject of a test on which the student failed to perform 
satisfactorily" (SBOE, 2001). It is the initial charge of the grade placement committee to provide some sort of 
accelerated instruction to prepare the child for the third test. After three attempts, the official code directs the school 
to retain the student. At this point in the process, the parent(s) or legal guardian may appeal to the grade placement 
committee to permit the child to move up to the next grade level. Citing from Official Code 20-2-283, "The grade 
placement committee may decide in favor of a student’s promotion only if the committee concludes, using standards 
adopted by the local board of education, that if promoted and given accelerated instruction, the student is likely to 
perform at grade level. A student may not be promoted on the basis of the grade placement committee's decision 
unless that decision is unanimous" (SBOE, 2001). 

Teachers Views on Social Promotion 



Governor Barnes' contention that teachers are critical of social promotion may have some credibility when the 
research is examined. Tomchin and Impara (1992) published a study showing eighty-two percent of elementary 
school teachers believed that retention helps children prevent future failure and seventy percent thought that the threat 
of failure motivates children to succeed. A whopping ninety-eight percent stated that they would never rule out the 
decision to fail a child (Tomchin & Impara, 1992). DiMaria (1999), in a study of New York City teachers, found 
similar results. In this 1999 study, sixty percent of teachers felt that students should never be socially promoted with 
thirty percent reporting that the primary grades were the best times to retain in grade. 



Clearly, an overwhelming majority of teachers feel that social promotion frustrates children by burdening them with 
schoolwork that is too advanced for them to comprehend. Teachers believe that this burden is an imposition, one that 
makes teaching much more difficult because it forces them to deal with the under-prepared while trying to teach those 
who are prepared (Thompson & Cunningham, 2000). The prevailing view of teachers is that instruction is easier and 
more effective when variability of academic competence within the class is reduced (Foster, 1993). 

Both of the teacher assumptions about student and teacher frustration, however, are not borne out by the educational 
research. It may be that teachers are not aware of the preponderance of retention research as evidenced by their 
reliance on anecdotal accounts from colleagues. Anecdotal reports by teachers often suggest that children benefit from 
retention, yet because of the decision to retain, there is no opportunity to see how well the children might have 
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progressed had they been promoted (NAECS, 2000). 



Mary Lee Smith wrote in Flunking Grades that teachers tend to access practical knowledge rather than formal 
knowledge. Practical knowledge is the sort of knowing that begins with personal experiences followed by future 
action based on these personal experiences. For example, one teacher remarked in Smith's study, "when my own son 
was retained, it was because he was too young for his age, and the next year he was a real leader in his class, and we 
never regretted that decision; and ever since then I have recommended that parents of young children in my class take 
the same step" (Smith, 1989, p. 133) 

Based on clinical interviews. Smith suggested that the educational philosophies of teachers fall into distinct categories 
and that these beliefs are directly related to their opinions on retention. One category Smith identified was designated 
Nativists, teachers who believe that the physiological maturation of ability develops over time in stages. Predictably, 
Nativist teachers feel that children should not be exposed to developmentally inappropriate instruction. If a teacher 
was not a Nativist, Smith found that teachers could be grouped in three additional ways. 1) Remediationists are 
teachers who are active instructional and resource managers, 2) Diagnostic Prescriptors believe that deficiencies, such 
as auditory memory and visual-motor problems, can be corrected when identified with specific instruction and 3) 
Interactionists, teachers who feci that successful teaching begins with the prior knowledge and interests that the child 
possesses (Smith, 1989). 

Smith found that the most likely to retain are Nativists because they are prone to see physical size and chronological 
age as reasons to hold children back. This is somewhat of a confounding finding given that Nativists’ beliefs are 
congruent with some widely held theories of child development (Smith, 1989). While Nativists are more likely to 
retain students than the others, all teachers agreed in Smith's study that retention is beneficial both in the short as well 
as the long term. Through her interviews Smith recorded anecdotes such as the yearning to put the child at the top of 
the class and many claims to the effect that there are no stigmas attached to retention if the teacher and parents handle 
it well. Often Smith heard about the disasters that occurred when children are socially promoted and very few teachers 
named any negative effects of retention. All stated that it is best to err on the side of retention and if any harm was 
done, its effects are temporary. Perhaps the most disturbing finding Smith reported is that teachers often discounted 
the child’s feelings of disappointment, failure and confusion or reluctantly acknowledged that if any emotional harm 
was done, the positives far outweigh the negatives. No one in Smith's research responded that social promotion was 
beneficial. Smith suggested that instructional efficiency pre-empts the child's best interest when she concluded, "The 
teacher is a self-interested theoretician. Though couched in the rhetoric of pupil benefits, her beliefs about retention 
are, perhaps unconsciously, conditioned by a wish for a more homogeneous and trouble free class" (Smith, 1989, p. 
149). 

Contradictions Between Research and Practice 

A preponderance of control-group studies, structured to measure the comparison between retained students and 
students recommended for retention but promoted anyway, come down clearly on the side of promotion. What these 
studies show is that students who are recommended for retention, but are nonetheless promoted to the next grade, end 
up doing just as well as or, in many cases, perform better academically than non-promoted peers (Foster, 1993; 
NAECS, 2000). 

While school performance is usually the focus of the debate, the most pernicious effect of retention is that the 
decision to fail a child usually results in dire social consequences. Children who have been retained demonstrate more 
social regression, display more behavior problems, suffer retention-related stress, and more frequently drop out of 
high school. (Grissom & Shepard, 1989; Frymier, 1997; NAECS, 2000). 



While it may seem unfair to some, keeping a child with his or her peer group is the best decision almost all of the 
time. When grade retention is used as a solution for poor performance, it is assumed that the problem resides in the 
child's learning ability. But, this is rarely the case (Darling-Hammond, 1998; Frymier, 1997). 

Shedding light on why children do not do well in school, studies show that the reasons for poor performance usually 
stem from non -academic factors such as a seriously ill parent or the death of a sibling or maybe a parent lost his or her 
job last year (Frymier, 1997). In addition, many students who failed a grade had been in an accident or were seriously 
ill during the year (Frymier, 1997). Now that the 2000 census has been published, data show that many non-English 
speaking children have become part of our schools (U.S. Census, 2001). Research tells us that if English is not spoken 
at home, a child is twice as likely to be retained during his or her schooling (Foster, 1993). Another suggestive 
statistic is that over half of the students who were retained in grade came from a broken home where moving from 
town to town was a frequent experience (Frymier, 1997). Concomitant with these factors is what research has coined 
as "retention bias," a tendency to retain a higher proportion of males, those with small physical stature, poor children 
and minority students (Foster, 1993; Frymier, 1997; Miller, 2001). 



Because repeating a grade is a highly visible act, one that separates a student from his age peers, what is most 
disturbing about failing a child is what happens to them afterwards. Rather than accepting failure, children perceive 
the decision to repeat a grade as a punishment for something out of their control, a perception that discourages them 
from completing school (Foster, 1993). It is well documented that students who are held back do worse in the long 
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run compared to students who are promoted, in part because they give up on themselves as learners (Denton, 2001). 
Even small children perceive that failing a grade is a serious social stigma. Stigmatizing children lowers their self- 
esteem, a psychological albatross that often results in a teen pregnancy or drug and alcohol use later on (Darling- 
Hammond, 1998; Foster, 1993). A review of sixty-six studies conducted between 1990 and 1997 found that sixty-five 
of them showed retention to be ineffective or harmful (Denton, 2001). In another 1997 study of twenty-three risk 
factors for school failure, students who fail a grade have many more problems, in every risk area, than those students 
who were promoted to stay with their peers (Denton, 2001; Frymier, 1997; Owens & Magliaro, 1998). 

Standardized Tests and Consequences of Failure 



In preparation for the implementation of the legislation that bans the practice of social promotion, the state of Georgia 
has administered for the second year a Criterion Referenced Competency Test (CRCT), commonly referred to as the 
State Curriculum Test, as a way to determine how many children might be retained in grade. While some reading 
scores have improved on average, about twenty-five percent of fourth, sixth and eighth grade children flunked the test 
(Georgia Department of Education, 2001). Looking at the best results, eighteen percent of fourth grade students failed 
the reading component. In the worst performing category, forty-one percent of eighth grade children failed to achieve 
a passing grade in mathematics (Georgia Department of Education, 2001). What is alarming is that, beginning in 
2003, promotion to the next grade will be based on the results of this test, meaning that one in five, maybe more, 
students will be retained in grade. While there is some clamoring to revise the CRCT so that more students will pass, 
using a standardized test to make the critical decision to pass or fail a student has questionable validity (Salzer, 2001). 
The technical complexity of performance-based standardized tests, coupled with the fact that performance based tests 
are relatively new assessment and evaluation techniques, means that tests like the CRCT require constant revision in 
their early stages (Elmore, Abelman & Furhman, 1996). These changes are evidence that there are fundamental flaws 
in performance based tests, changes that result in improvement in some 2001 CRCT scores over 2000 CRCT scores. 

Even though the practice of standardized testing is hard pressed to show that it can produce real gains in student 
learning, state policymakers count on test revisions to improve scores. With standardized testing, researchers have 
found that scores will initially be low and then rise for several years before leveling off This upward trend caused by 
the "saw tooth effect" is due to teacher-led test preparation rather than to student achievement (Miller, 2001). 
Predictably, policymakers tout the spike in test scores, usually within the second year of test introduction, as proof 
that accountability measures are working. 

While it is likely that the scores will get better because of this tinkering with the testing techniques and test 
preparation of students, there is no reliable evidence which suggests that performance-based tests will ever be 
perfected. Because the primary purpose of a standardized test is to gather data from a very large group of test takers as 
a way to evaluate if the overall curriculum needs to be improved, the CRCT should never be used to make a decision 
that affects an individual student (Miller, 2001). It is very important to note that a standardized test score does not 
reliably measure what an individual child actually knows because children are not consistent test takers. Even if the 
test was administered several times, the problem remains: snapshots cannot show a child's full range of capabilities 
(Kohn, 2000; Miller, 2001). 

Challenging the Claims of Success in Chicago, Baltimore, and Texas. 

With over fifty years of research showing that grade level retention provides no academic advantages to students, the 
practice of retention persists and is on the rise nationwide (Owings & Magliaro, 1998). The Consortium for Policy 
Research in Education (1990) reported that by the ninth grade, approximately fifty percent of all U.S. school students 
have been retained. If the goal of retention is to allow students more time to develop adequate academic skills so that 
they will be successful in subsequent years, why do the follow-up data on implemented programs throughout the 
United States show evidence to the contrary? 

Beginning in 1996, the Chicago public schools promoted only third, sixth, and eight graders who obtained the 
minimum score on the Iowa Test of Basic Skills Initial studies purporting to show the success of Chicago's program 
revealed that students, especially those with the lowest prior test scores, showed impressive gains after a full year of 
intervention and intensive summer instruction. However, follow-up studies revealed that learning gains dissipated 
after three years resulting in an increased likelihood of school drop out (Thompson & Cunningham, 2000; Denton, 
2001; Holmes, 1989). In Baltimore, a similar story unfolded; a study conducted at Johns Hopkins University found 
that the performance of students retained during elementary school did improve modestly during the year they 
repeated and for several years thereafter (Denton, 2001). Again, follow-up studies revealed that initial gains faded, 
with sixty-five percent of the retained students dropping out of school as compared to eighteen percent of all other 
students. For students who were held back more than once, the drop out rate soared to ninety-four percent (Denton, 
2001). What unfolds as one examines the research is that retained children are, on average, worse off than those who 
are socially promoted (Holmes, 1989; Shepard et al., 1996). 



As for the Texas model, the study claiming success had serious methodological shortcomings that limit its validity 
(Denton, 2001). Texas researchers reported in a 1999 study that the performance of retained third graders improved 
over those students who failed the Texas Assessment of Academic Skills test but were promoted to the next grade. 
The validity of the test comes into question due to the large disparity between the number of students who were 
retained (400 students) and the number who were promoted (35,000 students). The 400 students, one percent of the 



total, represented the extreme low end of the range of test scores, so any subsequent test scores had no place to go but 
up, a statistical phenomenon known as "regression to the mean." In addition to the test reporting flaws, Texas 
policymakers have also manipulated students to give the illusion of increased test scores. As reported by Haney 
(2000), school officials exclude poor test takers from the tenth grade TAAS by either retaining them in the ninth 
grade, classifying them as learning disabled, or encouraging them to leave school and pursue the GED. By employing 
these tactics, Texas schools can report apparent test score increases for the tenth grade students. 

A Snowball's Chance in Georgia: the viability of choosing litigation as a strategy to stop the use of 
high stakes tests to determine promotion or retention. 

Except for the mountains in the northern part of the state, it doesn’t snow very often in Georgia and, when it does, 
chances are that the frozen precipitation will not last very long, A snowball’s chance in Georgia has the life 
expectancy of a fruit fly, about a day. The same analogy holds true for using the courts to overturn the legislature’s 
decision to use the CRCT to determine promotion and retention. While some cases across the nation have been won 
on the local level, almost all have been overturned at the appellate level, meaning that victory in the courts is short- 
lived. Appellate courts have overturned challenges based on two cases. United States v. Fordice in 1992 and 
Personnel Administrator v. Feeney ini 979. Basically, these two cases frame the issue by deciding that "Placement 
testing, exit examinations, and achievement tests may be used to assist in the determination of classroom assignments 
and eligibility for graduation, provided that the test results are not a reflection of past racial segregation policies, the 
testing is accurate, and the results are open to public scrutiny" (Deskbook Encyclopedia of American School Law, 
2002, p. 489). Because the Equal Protection Clause of the Fourteenth Amendment forbids schools from engaging in 
intentional discrimination on the basis of race, color, national origin or sex , the first legal litmus test is whether or not 
a test perpetuates or preserves illegal discrimination. What the courts deemed important in Larry P. v. Riles in 1984 is 
that the State Department of Education had to foresee that the test would have a significant disproportionate impact by 
race. Second, the department of education has to have failed to show the validity of the test for minority children. 

Also, the test must cause a stigma and irreparable injury to the student. Moreover, Larry P. v. Riles demands proof 
that failing the test will result in effective educational opportunities for the child, (Heubert & Hauser, 1999). 



While the courts have recognized that high stakes tests have an adverse effect on minority children, they consistently 
reject the argument that these injuries are caused intentionally by the state. Instead, courts find that the state has a 
substantial governmental interest in education and that high stakes tests are a legitimate way to hold students 
accountable (Heubert & Hauser, 1999), Even when presented with clear accounts of racial bias, courts have refused to 
find that high stakes tests violate Title VI saying that they do not intentionally effect a particular race in an adverse 
manner (Heubert & Hauser, 1999; Deskbook Encyclopedia of American School Law, 2002). The Supreme Court 
forbids any practice that, while appearing to be a fair, perpetuates or promotes the effects of prior illegal segregation. 
This may mean that it is unlawful for any child who has attended an illegally segregated school at anytime in her/his 
schooling may not be subjected to a high stakes test designed for promotion or retention. It is rare today that a child 
has attended such a school, yet it opens the possibility that the courts could scrutinize a test more closely if a state or 
school district has had a recent history of segregation or intentional discrimination (Heubert & Hauser, 1999). 



The courts have almost uniformly dismissed claims of intentional discrimination and have steadfastly upheld that high 
stakes tests are rationally related to legitimate state interests (Deskbook Encyclopedia of American School Law, 

2002 ). 

While there is an abundance of research that shows that retention has deleterious effects, such as low self-esteem, 
negative attitudes toward school and a reduced chance at succeeding at school, the courts dismiss such reasoning as 
speculative. This view, that educational research is mere speculation, was evidenced through the case Erik v. by and 
through Catherine V.v. Causby North Carolina in 1997, a decision that upheld a school board’s decision to fail 
children based on a standardized test by rejecting the argument that students suffer irreparable harm when retained in 
grade because any potential harm is based solely on speculation. .In stark contrast to the educational research, the 
court rationalized the situation completely oppositely saying that because retention gives a child more time in school 
to catch up, the state is doing its job by providing more resources to those who need them (Deskbook Encyclopedia of 
American School Law, 2002), 

Texas courts also viewed retention as a part of a remediation benefit in the case, GI Forum v. Texas Education 
Agency in 2000. In this case, retention was affirmed as a part of a larger remediation process that provided those 
students who failed any portion of the exam with extra instruction intended to help them overcome their deficiencies. 
The court accepted the State’s position that school accountability and mandated remediation helped to address the 
effects of prior discrimination in Texas because the exam provided the state with an objective way to assess student 
mastery of the skills and knowledge. When the issue of racial and cultural bias was raised, the court ruled that the 
exam was not fundamentally unfair to minority students because it measured what it claimed to measure and what 
was tested was taught. Because the test was aligned with the curriculum, the court decided that it was a valid test that 
met accepted standards (Deskbook Encyclopedia of American School Law, 2002), 

What the states are relying upon in court is a conservative legal viewpoint that legitimizes high stakes testing for 
promotion so long as the tests comply with generally accepted standards for its use. These generally accepted 
standards have two central principles: 1 ,) a test score, like any other source of information about a student, is subject 
to error. Therefore, high stakes decisions like promotion should not be made automatically on the basis of a single test 



score (Shepard &Smith 1987; Darling Hammond & Falk, 1995) and 2.) a student’s test score on a test should be used 
only in conjunction with other information sources in making such an important decisions as promotion to the next 
grade (Heubert & Hauser, 1999, p. 126). The state is clearly skating on thin ice here given that the generally accepted 
practice standards among psychometricians do not support the use of standardized tests as stand alone instruments to 
determine grade level promotion. Take for instance, it is generally accepted practice to supplement test scores with 
other assessment measures such as those performed by the teacher in the classroom (Heubert & Hauser, 1999; AERA, 
1985, 1998; Joint Committee on Testing Practices, 1988). Moreover, there is legal precedence that could disrupt the 
states’ position demonstrated by the decision made in the United States v. Fordice in 1992, a ruling that rejected the 
use of one test score for placement decisions (Heubert & Hauser, 1999). 

A legal strategy that offers a glimmer of hope rests on the concept of disproportionality. What must be proven is that 
grade retention is disproportionate among protected minority groups when compared to whites and that this 
disproportionality will decrease if equally reliable alternative assessments are used (Heubert & Hauser, 1999). It is 
well documented that grade retention is disproportionate among blacks/hispanics when compared to whites by a 
margin of 2: 1 (Heubert & Hauser, 1999). The data shows that by ages 10-1 1 , ten percent more blacks and Hispanics 
are retained; by ages 15-17, forty to fifty percent more are retained and when students reach 15-17 years old fifty 
percent of blacks have fallen behind (Heubert & Hauser, 1999). There is a possibility that if the citizens of Georgia 
used disproportionality as a strategy, the state’s decision to use the CRCT could be ruptured when it is shown that 
those adversely affected are disproportionately protected minorities. Yet, any exuberance must be tempered with a 
good dose of healthy cynicism. 

In Texas, where the TAAS examination is used to determine promotion and retention, groups representing Texas 
minority students sued the state with the claim that the criterion referenced test discriminated against minority 
students in violation of the Due Process Clause of the 14th Amendment. What plaintiffs must show is that there is a 
preponderance of evidence that the policy of high stakes testing disproportionately has an adverse impact on a 
protected group, but this is not easy to determine. To prove adverse impact, proof must be presented via a study of the 
entire pool of test -takers that the success rate for members of a protected class is significantly lower than if a random 
sample was examined (Deskbook Encyclopedia of American School Law, 2002). Fortuitously for the children of rural 
declining Georgia, there is data available that has been compiled by the University of Georgia Department of Housing 
and Consumer Economics (2002) along with the State of Georgia’s Office of Education Accountability (2002) that 
meets the requirement for a study of the entire pool of test-takers showing that the success rate for members of a 
protected class is significantly lower than if a random sample was examined. 

Who Gets Hurt the Most; Relationships Between Poverty, Race and the Criterion Referenced 
Competency Test Results from Rural Declining Georgia. 

As a way to illuminate just how pernicious a law such as this will be to the poorest among us, we have conducted a 
study of school systems in the thirty-nine counties categorized as "declining rural counties" in Georgia, commonly 
referred to as the "black belt’’, so named because of the large number of African Americans who reside in them. 

Our methodology compared county by county demographic data compiled by the University of Georgia Department 
of Housing and Consumer Economics (University of Georgia Department of Housing and Consumer Economics, 
2002) with the State of Georgia's Office of Education Accountability's (State of Georgia's Office of Education 
Accountability, 2002) statistics for each district. Because there is compelling evidence that family background is the 
primary determinate for school achievement (Shepard & Smith, 1989; Elmore, Abelman & Furhman, 1996; Clotfelter 
& Ladd, 1 996), our study includes an analysis of eight socio-economic categories; 1 ) percentage of population that is 
African American, 2) per capita income, 3) children in poverty, 4) African Americans in poverty, 5) female headed 
families in poverty, 6) un-wed births, 7) percentage of population without a high school diploma, and 8) percentage of 
African Americans without a high school diploma. Given that family background is such an important predictor of 
success, it is critical to supplement the school lunch index, the common statistic used to determine poverty in schools, 
with multiple economic and cultural measures. 



When we first embarked on this study, two objectives were foremost: 1 ) compare the data gathered from these thirty- 
nine rural declining counties with statewide data; 2) present descriptive statistics that illuminate the relationship 
between CRCT scores and multiple socio-economic data. But, after we looked closer at the numbers, we discovered 
that in many of these counties, the school district data did not match up with countywide data. After comparing the 
county population demographics with the school systems data, it became apparent that many of these school systems 
have a discemable racial imbalance. Because this discovery suggests that race will matter in the decision to fail a 
child in Georgia, this research was expanded to include a discussion about the future of the rural African American 
community once the CRTC is implemented. 

These thirty-nine counties form a constellation of poverty that slashes through the southern region of the state of 
Georgia. Forming contiguous pockets of counties in rural decline, the constellation extends in a chain from the far 
southwest comer to the eastern part of the state. As a way to boost the clarity of the research, we have chosen to 
present this data through a geographic journey whose starting point begins in the most concentrated area of poverty in 
southwest Georgia. Traveling across the state, this study will explore those counties that make up the belt buckle, a 
band that traverses along the mid-section of the state from west to east, followed by a discussion of those counties that 
are located in the east. 
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Southwest Rural Declining Counties 



These twelve counties are found huddled along the Alabama and Florida border in the farthest southwest comer of the 
State of Georgia framed by the Chattahooche River to the west, Albany, Georgia as the closest city to the east and 
Columbus, Georgia to the north. There are no major roads cutting through nor are there towns of any substantial 
population. While there may never be an occasion for many travelers to ever pay the folks here a visit, these twelve 
counties are home to 92,400 Georgians, of which, 14,080 are children in the public school system. 



Table 1 

Southwest Rural Declining Counties 
SES Data Compared to Georgia State SES Data 



SES Attributes 


Range in Southwest Rural Declining County Data 


State Data 


Population African American 


47.0% to 61.5% 


28.0% 


1 999 Per Capita Income 


$16,153 to $22,270 


$27,324 


1997 Children in Poverty 


27.2% to 47.4% 


21.8% 


1989 African American in Poverty 


33.7% to 53.0% 


30.3% 


1989 Female Headed Families in Poverty 


34.0% to 70.2% 


34.3% 


1999 Unwed Births 


42.9% to 65.9% 


36.6% 


No High School Diploma 


46.4% to 60.9% 


29.1% 


African American No High School Diploma 


56.6% to 70.6% 


41.4% 



As Table 1 shows, these twelve counties have two to three times more African American citizens than the rest of the 
state, most of whom live in poverty. Because school children here are likely to be poor and living in a household 
headed by an unwed mother who dropped out of school, the prospect for academic success is bleak. With up to 
seventy percent of the African American population lacking a high school degree, academic role models are hard to 
come by. When Table 2 is examined, it becomes apparent that all of these counties have reported similar socio- 
economic data. 



Table 2 

Southwest Georgia Counties in Rural Decline Socio-Economic Status 



County 


Pop. African 
American 


1999 Per 
Capita 
Income 


1997 

Children in 
Poverty 


1989 African 
American in 
Poverty 


1989 Female 
Headed Families 
in Poverty 


1999 

Unwed 

Births 


No High 
School 
Diploma 


African American 
No High School 
Diploma 


Baker 


50.4% 


$20,940 


35.7% 


35.0% 


41.1% 


41.7% 


46.4% 


56.6% 


Calhoun 


60.6% 


$21,646 


39.9% 


46.0% 


48.4% 


55.2% 


58.1% 


58.1% 


Clay 


60.5% 


$17,082 


46.9% 


50.6% 


52.3% 


57.8% 


48.6% 


63.4% 


Early 


48.1% 


$21,115 


42.2% 


51.5% 


61.7% 


57.0% 


45.9% 


62.4% 


Miller 


28.9% 


$22,270 


34.8% 


37.2% 


41.2% 


52.8% 


42.6% 


63.6% 


Mitchell 


47.9% 


$21,392 


35.8% 


45.3% 


53.8% 


57.0% 


45.1% 


63.5% 


Quitman 


46.9% 


$18,223 


47.4% 


52.3% 


70.2% 


63.6% 


50.5% 


68.8% 


Randolph 


59.5% 


$18,298 


43.3% 


53.0% 


55.8% 


57.3% 


50.7% 


64.6% 


Seminole 


34.7% 


$19,247 


39.3% 


47.5% 


48.5% 


49.7% 


48.6% 


60.9% 


Stewart 


61.5% 


$18,744 


38.0% 


45.2% 


56.4% 


48.6% 


60.9% 


70.6% 


Terrell 


60.7% 


$16,153 


38.7% 


42.4% 


51.1% 


65.9% 


47.6% 


65.3% 


Webster 


47.0% 


$20,728 


27.2% 


33.7% 


34.0% 


42.9% 


49.6% 


65.2% 


State 


28.0% 


$27,324 


21.8% 


30.3% 


34.3% 


36.6% 


29.1% 


41.4% 



A child attending school in these counties would have a one in three, at best a one in five, chance of passing the the 
third grade to the fourth grade once the CRCT decides his or her fate, a probability much worse than the rest of the 
state (Table 3). 

Passing on to the sixth grade will be even more difficult, given that your odds are about 50/50 that you will pass the 
CRCT. While the scores statewide are improving in the sixth grade CRCT, school's scores are getting worse, ever 
widening the gap between rich and poor. If a child is so fortunate as to make it to the eighth grade in 2006, chances 
are better than even that he or she will not go to high school the next year because they failed the mathematics portion 
of the CRCT. As for comparing their school to the rest of the state (Tables 3 & 4), their school is in a free falling 
spiral, dropping significantly behind an abysmal statewide percentage of failing students. 
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Table 3 



BEST COPY 



Southwest Georgia Rural Declining Counties 



Percent Failing CRCT 4th Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


Southwest Counties 


32% 


31% 


50% 


Statewide 


26% 


26% 


38% 


Percent Change 


+23% 


+19% 


+32% 



Percent Failing CRCT 6^^ Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


Southwest Counties 


37% 


47% 


46% 


Statewide 


24% 


36% 


31% 


Percent Change 


+54% 


+31% 


+48% 



Percent Failing CRCT 8^^ Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


Southwest Counties 


30% 


44% 


56% 


Statewide 


18% 


32% 


41% 


Percent Change 


+67% 


+38% 


+37% 



Table 4 

African American Southwest Georgia Rural Declining Counties 



Percent Failing CRCT 4^^ Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


African American Southwest Counties 


37% 


35% 


57% 


African American Statewide 


37% 


34% 


52% 


Percent Change 


0 


+3% 


+10 



Percent Failing CRCT 6^^ Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


African American Southwest Counties 


38% 


47% 


50% 


African American Statewide 


35% 


49% 


45% 


Percent Change 


+9% 


-4% 


+11% 



Percent Failing CRCT 8^^ Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


African American Southwest Counties 


36% 


51% 


64% 


African American Statewide 


27% 


45% 


58% 


Percent Change 


+33% 


+ 13% 


+10% 



With the exception of one of the twelve counties, Webster County, the racial balance of the schools when compared to 
the general population is egregiously disproportional. Calhoun County’s population is sixty percent African American, 
yet Calhoun County Schools have too few whites to report, meaning that forty percent of the white children in 
Calhoun County attend private schools or are home-schooled. Sixty percent of the fourth graders, forty-four percent 
of the sixth graders and sixty percent of the eighth graders in Calhoun County failed at least one CRCT content area 
test. 



Terrell County is the same story, only worse. Terrell is also sixty percent African American with no significant white 
representation in the schools. Having the lowest per capita income of around $16,000, Terrell County will face the 
fact that sixty-two percent of the fourth graders, sixty percent of the sixth graders and sixty-eight percent of the eighth 
graders will fail their respective grades. 




Quitman County, where forty-six percent of the population is African American, has no whites attending the one 
elementary public school there. In Quitman County, teachers and principals will face the daunting responsibility for 






carrying out the failure sentence for eighty-one percent of the fourth grade class and sixty-five percent of the sixth 
grade. 



The same situation obtains in Randolph County, with ninety-three percent of their children on the free lunch program; 
teachers and principals there will be forced to fm\ sixty percent of the fourth, sixth and eighth grade students. While 
Clay County CRCT scores are not as low as the others, this all African American school system, with a per capita 
income of $17,000 and sixty-five percent of African Americans in the county without a high school degree, will fail 
twenty-nine percent in the fourth grade, fifty-five percent in the sixth grade and because forty-eight percent of the 
eighth grade did not meet the mathematics standards of the CRCT, they too, will fail. The remaining schools in the 
counties. Baker, Early, Miller, Mitchell, Seminole and Stewart, are also disproportionately African American when 
compared to the general population. Most of the schools in these counties are two-thirds African American with 
county data showing a range of one-third to one-half of the population as African American. 

Mid-State Rural Declining Counties 

Fourteen counties form a contiguous swath of land beginning in Talbot County, situated between Columbus and 
Macon Georgia, southward along Interstate 75 to the Florida border, where Clinch and Ware Counties envelope the 
great Okeefenokee Swamp. These mid-state counties are home for 1 68,276 Georgians, 28,854 of whom are children 
in the public schools. 



Table 5 paints a picture of economic and social crisis with data that shows per capita income well below the state 
average, resulting in significantly more children in poverty. As with the southwestern counties, school children in the 
mid-state counties are likely to have a parent who is a poor, unwed African American mother without a high school 
diploma. 



Table 5 

Mid-State Rural Declining Counties 
SES Data Compared to State 



SES Attributes 


Range in Midstate Rural Declining County Data 


State Data 


Pop. African American 


24.6% to 61.6% 


28% 


1999 Per Capita Income 


$15,585 to $23,202 


$27,324 


1997 Children in Poverty 


26.7% to 38.9% 


21.8% 


1989 African American in Poverty 


34.7% to 57.8% 


30.3% 


1989 Female Headed Families in Poverty 


37% to 64.7% 


34.3% 


1999 Unwed Births 


37.7% to 62.7% 


36.6% 


No High school Diploma 


39.4% to 53.8% 


29.1% 


African American No High School Diploma 


54% to 69.8% 


41.4% 



When each county is examined separately in Table 6, the relationship between race, poverty and educational 
attainment becomes clearer. Dooley, Macon, and Talbot, counties with the largest African American populations are 
the poorest; while Bleckley, Irwin, and Pulaski counties, with many fewer African Americans, are better off. These 
data suggest that this economic divide persists because of the lack of educational attainment among African 
Americans. When the column "African American No High School" is examined in Table 6, the data describe a 
population that has, for the most part, found it difficult to graduate from high school. Ten of these fourteen counties 
have anywhere from sixty to seventy percent of the African American population without a high school degree; the 
remaining four counties have fifty to sixty percent without a diploma. 



Table 6 

Mid-State Counties in Rural Decline Socio-Economic Status 




County 


Pop. 

African 

Am. 


1999 Per 
Capita 
Income 


1997 Children 
in Poverty 


1989 African 
Am. in Poverty 


1989 Female 
Headed Families in 
Poverty 


1999 

Unwed 

Births 


No High 
school 
Diploma 


African Am. No 
High School 
Diploma 


Bleckley 


24.6% 


$21,771 


26.7% 


42.0% 


37.0% 


41.8% 


39.7% 


66.7% 


Clinch 


29.5% 


$18,379 


32.9% 


40.9% 


53.6% 


49.5% 


53.8% 


62.6% 


Cook 


29.1% 


$18,276 


30.4% 


39.2% 


42.7% 


37.7% 


44.8% 


54.0% 


Dooley 


49.5% 


$18,690 


37.0% 


50.9% 


55.6% 


54.1% 


45.3% 


60.1% 


Irwin 


25.9% 


$20,832 


29.8% 


47.9% 


50.3% 


40.0% 


46.9% 


62.4% 


Lanier 


25.6% 


$17,675 


34.1% 


34.4% 


54.2% 


39.0% 


48.8% 


62.1% 


Macon 


59.5% 


$19,927 


37.1% 


39.8% 


49.3% 


62.7% 


46.3% 


57.3% 
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Pulaski 


34.3% 


$23,202 


29.5% 


48.5% 


50.1% 


53.9% 


39.4% 


64.2% 


Taylor 


42.6% 


$18,774 


38.2% 


49.0% 


57.6% 


47.7% 


48.8% 


69.8% 


Telfair 


38.4% 


$18,477 


35.2% 


41.2% 


49.9% 


45.3% 


47.9% 


65.7% 


Turner 


41.0% 


$17,831 


38.6% 


57.8% 


64.7% 


53.5% 


44.7% 


64.3% 


Talbot 


61.6% 


$15,385 


34.1% 


34.7% 


46.4% 


48.8% 


43.8% 


58.2% 


Ware 


28.0% 


$19,738 


33.7% 


42.5% 


41.7% 


45.2% 


38.9% 


51.4% 


Wilcox 


36.2% 


$19,834 


38.9% 


56.2% 


61.4% 


51.2% 


47.2% 


69.6% 


State 


28.0% 


$27,324 


21.8% 


30.3% 


34.3% 


36.6% 


29.1% 


41.4% 



The socio-economic divide between Whites and African Americans in the declining rural counties of Georgia will 
surely be exacerbated through the implementation of the CRCT mandates. Because of retention in the third, fifth and 
eighth grades, African American children will be systematically encouraged eventually to drop out of school, 
resulting in the inability to command wages that might lift them out of poverty. Supporting the assumption that 
income is proportional to test scores, Tables 6 & 7 suggest that there is a relationship between CRCT test scores and 
the income earned by African Americans. When Mid-State African American CRCT scores are compared to 
statewide figures, the data show that poorer African Americans living in rural declining counties do worse than those 
African Americans who live in counties with higher income levels. Further, there is evidence that African American 
students who live in places where more of the African American population has earned a high school diploma do 
better on the CRCT than counties with less educational attainment. 

Table 7 

African American Mid-State Georgia Rural Declining Counties 



Percent Failing CRCT 4^^ Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


African American Mid-State Counties 


48% 


44% 


61% 


African American Statewide 


37% 


34% 


52% 


Percent Change 


+30% 


+29% 


+ 17% 



Percent Failing CRCT 6^^ Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


African American Mid-State Counties 


49% 


59% 


48% 


African American Statewide 


35% 


49% 


45% 


Percent Change 


+40% 


+20% 


+7% 



Percent Failing CRCT 8^*^ Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


African American Mid-State Counties 


34% 


49% 


64% 


African American Statewide Counties 


27% 


45% 


58% 


Percent Change 


+26% 


+9% 


+ 10% 



The poverty to failure equation is repeated in Table 8 when the aggregate Mid-State CRCT scores show much lower 
results than the statewide data. 



Table 8 

Mid-State Rural Declining Counties 



Percent Failing CRCT 4*^ Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


Mid-State Counties 


36% 


33% 


45% 


Statewide 


26% 


26% 


38% 


Percent Change 


+36% 


+27% 


+ 18% 



o 

ERIC 



Percent Failing CRCT 6^^ Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


Mid-State Counties 


33% 


46% 


34% 



410 



Statewide 


24% 


36% 


31% 


Percent Change 


+37% 


+28% 


+10% 



Percent Failing CRCT Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


Mid-State Counties 


25% 


39% 


48% 


Statewide 


18% 


32% 


41% 


Percent Change 


+39% 


+22% 


+ 17% 



Eastern Georgia Rural Declining Counties 

Consider the Eastern Rural Declining Counties. These fourteen counties stretch vertically southward from counties 
that lie northwest of Augusta to rural areas southwest of Savannah. While these counties have very similar socio- 
economic data commensurate with very low CRCT test scores (Tables 9 and 10), some data are particularly 
noteworthy. Screven County's eighty percent mathematics failure in the eighth grade for African American students 
means that eight out of ten African Americans will not go to high school once the CRCT becomes the arbiter for 
promotion. Hancock County's statistics show that eighty percent of the children bom in 1999 in the county live in 
single parent households, meaning that the kindergarten class in 2004 would have eight out of ten children living with 
a single parent. 



Table 9 

Eastern Rural Declining Counties 



Percent Failing CRCT 4^*^ Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


Mid-State Counties 


38% 


38% 


49% 


Statewide 


26% 


26% 


38% 


Percent Change 


+46% 


+46% 


+29% 



Percent Failing CRCT 6^*^ Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


Eastern Counties 


34% 


47% 


37% 


Statewide 


24% 


36% 


31% 


Percent Change 


+42% 


+31% 


+19% 



Percent Failing CRCT Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


Eastern Counties 


27% 


46% 


51% 


Statewide 


18% 


32% 


41% 


Percent Change 


+50% 


+44% 


+24% 



Table 10 

African American Eastern Georgia Rural Declining Counties 



Percent Failing CRCT 4^*^ Grade 



Contei'it Ai'ca 


Reading 


English/ Language Arts 


Mathematics 


African American Eastern Counties 


47% 


45% 


61% 


African American Statewide 


37% 


34% 


52% 


Percent Change 


+27% 


+32% 


+17% 



Percent Failing CRCT 6^*^ Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


African American Eastern Counties 


44% 


57% 


47% 


African American Statewide 


35% 


49% 


45% 




411 



Percent Change 



+25% +16% +4% 



Percent Failing CRCT 8^^ Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


African American Eastern Counties 


35% 


54% 


63% 


African American Statewide Counties 


27% 


45% 


58% 


Percent Change 


+30% 


+20% 


+9% 



When Tables 1 1 and 12 are compared to the southeastern and mid-state counties, there is a distinct outlier, Glascock 
County, a county with a small African American population of 8.3%, which is not consistent with the data from the 
other thirty-eight counties. Located in the center of a chain of five rural declining counties, Glascock County stands 
out as the only rural declining county that has SES and CRCT data better than, or comparable to, the State averages 
(Table 13). Because Glascock County’s per capita income is in line with the other rural declining counties, the 
variable that confounds the repeated pattern of poverty and low CRCT scores is whiteness (Tables 14 &1 5). Not only 
will the vast majority of Glascock County students be promoted, about one third of the CRCT test takers actually 
exceeded the standards, a statistic not seen in any of the other thirty-eight county data. When the data from the other 
four counties in the chain are compared, Glascock County’s relatively low unwed birth rate appears to reduce the 
number of children in poverty, suggesting that the ’’children in poverty” and ”un-wed mothers” statistics may also be 
predictors for CRCT achievement. 



Glascock County’s segregation from its neighbors leads to another assumption germane to this research, that the 
CRCT Test creates a new kind of discrimination - one that hides behind the appearance of fair testing to mask 
persistent inequalities in the quality of education that rural African American children receive in Georgia (McNeil, 
2000). Walter Haney, of Boston College’s Center for the Study of Testing, warns that, ’’The consequences of 
standardized tests for Black and Hispanic students are clearly criminal from an educational point of view. It remains 
to be seen whether they are criminal under the United States Constitution” (McNeil, 2000, p. 231). 

Table 11 

Eastern Georgia Rural Declining Counties SES Data Compared to State 



SES Attributes 


Range in Eastern Rural Declining County Data 


State Data 


Pop. African American 


8.3% - 77.8% 


28% 


1999 Per Capita Income 


$16,787-$21,565 


$27,324 


1997 Children in Poverty 


22.3% -45.4% 


21.8% 


1989 African American in Poverty 


25.8% - 54.4% 


30.3% 


1989 Female Headed Families in Poverty 


29.4% - 64.2% 


34.3% 


1999 Unwed Births 


29.4% - 80.6% 


36.6% 


No High school Diploma 


38% - 57.2% 


29.1% 


African American No High School Diploma 


54% - 80.6% 


41.4% 



Table 12 

Eastern Georgia Counties in Rural Decline Socio-Economic Status 



County 


Pop. 

African 

Am. 


1999 Per 
Capita 
Income 


1997 Children 
in Poverty 


1989 African 
Am. in 
Poverty 


1989 Female 
Headed Families in 
Poverty 


1999 

Unwed 

Births 


No High 
school 
Diploma 


African Am. No 
High School 
Diploma 


Emanuel 


33.3% 


$18,336 


36.6% 


46.1% 


43.5% 


54.6% 


47.4% 


65.7% 


Glascock 


8.3% 


$19,496 


22.3% 


29.6% 


29.4% 


27.3% 


49.7% 


80.6% 


Hancock 


77.8% 


$16,787 


37.4% 


33.8% 


49.6% 


80.6% 


50.5% 


54.8% 


Jefferson 


56.3% 


$17,673 


36.3% 


38.3% 


41 .6% 


58.7% 


48% 


59.4% 


Jenkins 


40.5% 


$18,174 


37.4% 


46.8% 


53% 


57.6% 


50.1% 


71.2% 


Johnson 


37% 


$18,845 


36.3% 


38.3% 


41.6% 


58.7% 


48% 


59.4% 


Screven 


45.3% 


$19,181 


31.9% 


37.6% 


52.5% 


55.3% 


41.1% 


56.2% 


Taliaferro 


60.3% 


$17,383 


45.4% 


44.1% 


52.5% 


67.9% 


51.4% 


65.6% 


Tatnall 


31.4% 


$19,943 


34.6% 


42.7% 


50.3% 


44.3% 


42.6% 


56.9 


Treutlen 


33.1% 


$16,499 


37.5% 


47.3% 


48.2% 


47.1% 


47.3% 


56.4% 


Warren 


59.5% 


$17,664 


39.9% 


44% 


56% 


69.7% 


57.2% 


68% 


Wheeler 


33.2% 


$18,864 


37.9% 


54.4% 


64.2% 


32.5% 


43.3% 


54% 






















Wilkenson 


40.7% 


$19,614 


27.1% 


25.8% 


39.7% 


45.7% 


38% 


48.4% 


Wilkes 


43.1% 


$21,565 


29.4% 


37.1% 


40.5% 


50.8% 


43.4% 


62% 


State 


28.0% 


$27,324 


21.8% 


30.3% 


34.3% 


36.6% 


29.1% 


41.4% 



Table 13 

Glascock, Hancock, Jefferson, Taliaferro, and Warren Counties SES Data 



County 


Pop. 

African 

Am. 


1999 Per 
Capita 
Income 


1997 Children 
in Poverty 


1989 African 
Am. in 
Poverty 


1989 Female 
Headed Families in 
Poverty 


1999 

Unwed 

Births 


No High 
school 
Diploma 


African Am. No 
High School 
Diploma 


Glascock 


8.3% 


$19,496 


22.3% 


29.6% 


29.4% 


27.3% 


49.7% 


80.6% 


Hancock 


77.8% 


$16,787 


37.4% 


33.8% 


49.6% 


80.6% 


50.5% 


54.8% 


Jefferson 


56.3% 


$17,673 


36.3% 


38.3% 


41.6% 


58.7% 


48% 


59.4% 


Taliaferro 


60.3% 


$17,383 


45.4% 


44.1% 


52.5% 


67.9% 


51.4% 


65.6% 


Warren 


59.5% 


$17,664 


39.9% 


44% 


56% 


69.7% 


57.2% 


68% 



Table 14 

Glascock, Hancock, Jefferson, Taliaferro, and Warren Counties CRCT Data 



Percent Failing CRCT 4^^ Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


Glascock 


21% 


27% 


30% 


Hancock 


33% 


35% 


62% 


Jefferson 


46% 


43% 


56% 


Taliaferro 


60% 


57% 


65% 


Warren 


49% 


35% 


75% 



Percent Failing CRCT 6^^ Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


Glascock 


23% 


34% 


20% 


Hancock 


40% 


51% 


41% 


Jefferson 


37% 


51% 


43% 


Taliaferro 


42% 


75% 


46% 


Warren 


60% 


67% 


66% 



Percent Failing CRCT 8^*^ Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


Glascock 


11% 


22% 


19% 


Hancock 


25% 


42% 


66% 


Jefferson 


35% 


53% 


63% 


Taliaferro 


47% 


59% 


76% 


Warren 


19% 


39% 


63% 



Table 15 

African American Glascock, Hancock, Jefferson, Taliaferro, and Warren Counties CRCT Data 



Percent Failing CRCT 4^*^ Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


Glascock 


Too Few to Report 


Too Few to Report 


Too Few to Report 


Hancock 


32% 


34% 


62% 


Jefferson 


54% 


50% 


65% 


Taliaferro 


61% 


56% 


67% 


Warren 


46% . 


34% 


74% 




Percent Failing CRCT 6^^ Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


Glascock 


Too few to report 


Too few to report 


Too few to report 


Hancock 


40% 


51% 


41% 


Jefferson 


45% 


58% 


49% 


Taliaferro 


48% 


81% 


48% 


Warren 


60% 


66% 


65% 



Percent Failing CRCT 8^^ Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


Glascock 


Too few to report 


Too few to report 


Too few to report 


Hancock 


26% 


43% 


67% 


Jefferson 


36% 


56% 


67% 


Taliaferro 


50% 


63% 


81% 


Warren 


19% 


39% 


63% 



Giving Up on Going to High School 



Overwhelmingly, African Americans in Rural Declining Counties are at much greater risk of failing the fourth, sixth 
and eighth ^ade CRCT than African Americans who live in cities, suburbs or rural growth counties. Yet, those most 
at risk are eighth graders who attend all-Black schools in the rural declining counties. Because they failed one or more 
of the content area tests, chances are that most of the eighth graders in these ten counties will not go on to high school. 
Clearly, the worst performing category was mathematics with only forty-four percent eligible to move up to the ninth 
grade. If the law were effective today, Taliaferro County would send four students to high school leaving thirteen 
behind, Talbot County would send only eleven, holding back forty-four. Calhoun fails thirty-three of their fifty-seven 
eighth graders. Clay County retains half of their thirty-two children. In the larger counties, Hancock retains eighty of 
one hundred twenty-one as Terrell County keeps ninety-one of their one hundred thirty-three eighth graders in middle 
school. Ten of the thirty-nine Rural Declining Counties fall into the 'all-Black' school system category with six of 
these, Calhoun, Quitman, Clay, Randolph, Stewart and Terrell, being located in the Southwest section of the State. 

The others, Taliaferro, Hancock and Warren surround all white Glascock County in the east with Talbot County being 
the lone all-Black school district in the Mid-State region. 



Because each of these counties has its own school district, these schools are not considered illegally segregated. 
Drawing school district lines by county does not, superficially at least, appear to be gerrymandering given that each 
school district corresponds to an established county. Yet, segregation is, nonetheless, the result and the children in 
these schools suffer all the doi sad vantages of a segregated education. 

Future Consequences 

Being the most impoverished counties in the state, these Rural Declining school systems faced formidable challenges 
before the legislation to end social promotion was passed. With an average of twenty-five percent of their populations 
under the age of seventeen, these counties have large numbers of children who need enormous resources to overcome 
obstacles to academic success. What makes the "Declining Rural Counties" of Georgia's plight unique is that the 
children who attend schools in these counties will be denied de facto their property rights to a public education when 
they drop out of school as a result of the practice of grade retention. From the data presented here, most of these 
counties already have very high drop out rates. What percentage will the drop out rate reach when thirty-five to fifty 
percent of all fourth, sixth and eighth graders will be retained in grade? The question that ought to be asked is "Is this 
legislation really intended to improve education or is it a strategy to reduce the State's financial obligation to the rural 
poor?" It is clear that failing masses of poor children will not improve pedagogy because punishing children with 
retention does not change teaching. What we do know is that the association between retention and dropping out is 
noted consistently throughout educational research. Without a doubt, flunking children increases the risk of dropping 
out of school (Frymier, 1997). Because these thirty-nine counties are very poor, and the tax base available for public 
schools is small, the State of Georgia compensates for this revenue deficiency by making exceptionally large 
contributions to these counties. Thus, while not stated as policy, it cannot be ignored that the CRCT will most likely 
save the State a considerable amount of money by reducing the number of students in schools in these counties. 



While we dispute claims that the CRCT is a valid instrument to determine if a child should be retained in grade, we do 
not dispute that the CRCT is ironically a very reliable measure of economic resources (Kohn, 2001). As legislators 
extol the virtues of achieving academic excellence by using a "fair" test, like the CRCT, to determine if a child passes 
or fails, some wonder if the real agenda is a subtle form of class warfare intended to institutionalize intergenerational 
immobility and social stratification (Ohanian, 1999), a kind of violence that leaves behind the children with the least 
resources (Spring, 2000). Given the correspondence between the economic system and the role that the institution of 



education plays in perpetuating the class stratification of society (Bowles & Gintis, 1976), this legislation guarantees 
that the social reproduction of the society in rural Georgia will be preserved. The claim that flunking these children is 
for their own good is unconscionable, considering the dire social and economic punishments that will be imposed 
upon them. Ohanian derided such accountability measures "as cynical as handing out menus to homeless people in the 
name of eradicating hunger" (Ohanian, 1999, p. 31). 

Chelsea^s Story: Putting a Face on the Data 

We first met Chelsea at the garage where we have our car fixed; it was about 3:30 p.m. on a sweltering spring day in 
rural South Georgia. Her great-grandmother had just picked her up from school to bring her to the garage to finish out 
the workday. Great-grandmother sat down at her desk where she worked as the receptionist and bookkeeper for the 
family owned garage. As I waited for my car to be repaired, we both asked the light brown skinned six-year-old with 
big brown eyes and braided black hair if she enjoyed her day at school. Chelsea smiled broadly, "Yes!" she 
exuberantly replied as she showed us the cover of a book she brought home. From all appearances, Chelsea was a 
healthy, well-loved first grader. But, this has not always been the case for Chelsea. Just recently, her great- 
grandmother took legal measures to have Chelsea taken from her mother after a man who was living in the home 
broke Chelsea's arm while trying to spank her. After he broke Chelsea's arm, the boyfriend badly burned her baby 
brother because he was crying too loudly. This abuse by one of Chelsea's mother's boyfriends was one of many 
horrors that this beautiful little girl had to endure over the course of her short life. As great-grandmother tells it, her 
grandson had a short affair with Chelsea's mother, an affair that brought Chelsea into the world. After her grandson 
left Chelsea's mother, great-grandmother felt obligated to support Chelsea in any way that she could. The offer to give 
financial support and emotional support was a selfless act of love given that great-grandmothers' extended family is 
barely making ends meet. In addition to the financial hardship, it was a painful situation for great-grandmother to 
witness the cruelty inflicted on Chelsea. Chelsea's mother was involved with too many men and had too many 
substance addictions to care for her children. While great-grandmother reported to the social service system many 
times that Chelsea was being neglected and abused, Chelsea remained with her mother. It took an act of violence 
against the children for the system to intervene on Chelsea's behalf Even after the occurrence of such crimes 
committed against helpless children, Chelsea must still comply with a visitation plan that includes overnight stays 
with her mother. 



Behind in School. After a conversation with great-grandmother later in the Spring, we became aware that Chelsea did 
not do very well academically during the year. As a result of her poor academic performance, due to excessive 
absences in the beginning of the year, Chelsea's teacher recommended that she repeat the grade. Already retained once 
in kindergarten, Chelsea's great-grandmother was reluctant to hold her back again, great-grandmother felt that by 
Chelsea now being in a loving and secure home, the prospect that she will catch up to her peers is promising. 



Chelsea's teacher is basing her recommendation to retain on the failing grades that Chelsea received during the school 
year. Yet, if the Iowa Test of Basic Skills and a psychological assessment conducted at the behest of the Department 
of Family Services are considered in the decision, it is clear that Chelsea's poor grades were the result of poor 
attendance rather than cognitive ability. While Chelsea was absent for most class time when she lived with her 
mother, her scores were not as low on the ITBS as one might expect, and in some linguistic areas, her scores were 
high. As a result of high scores in certain verbal skills, Chelsea scored a composite word analysis in the sixty-fifth 
percentile. Chelsea's overall math abilities were average, but her composite math score was low because of a very low 
score on the computation section of the test. 



In support of the decision to promote, Chelsea scored at a grade level of 2.2 in reading and 2.0 in math on the 
Kaufman Test of Educational Achievement in a low stress testing environment. The Kaufman Test showed that 
Chelsea is beginning to read independently and can add and subtract simple numbers. As an indicator of her mental 
ability, the Weschler Intelligence test showed that Chelsea is a child of average intelligence. The psychologist who 
observed Chelsea noted that she was a cheerful child with no signs of depression. The drawings she drew for the 
psychologist showed a loving family situated around a sturdy tree and a solid house. The psychologist concluded that 
Chelsea was now in a loving family that could be trusted for support. 



The assessment went on the note that Chelsea is not a behavioral problem in school, she has many friends and is 
openly curious about new things. She is articulate, attentive, with good concentration and perseverance. Chelsea does 
not have any violent tendencies and, it is worth noting, that in light of her own physical abuse, she is quoted as saying 
that it is wrong to hurt anyone else. Having a birthday in October of 1993 makes Chelsea one of the oldest in her first 
grade class. If her great-grandmother had not insisted that Chelsea be promoted, holding her back again may have 
compounded her risk for dropping out of school later in life. Chelsea is progressing very well in the second grade and 
her chances for success are bright. Yet, another story could have unfolded if a standardized test was the arbiter for her 
future. 

Preventing Failure with Changes in Education Policy 

The fact that many school systems are overwhelmed by the increased number of under prepared students and use 
social promotion as a necessity, the public backlash should not be directed to testing and retention as the answers. 
Retention policies should be highly suspect given the lack of demonstrated effectiveness and prevalent bias against 
certain groups of children. Focusing public attention on standardized testing directs attention away from input issues 
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that can substantially effect quality education and places the blame instead on students and teachers. Because the 
negative effects of failure on children's achievement, motivation, self-concept, and graduation rates are so well 
known, one of the most important decisions in a young person's life should not be based on the outcome of a 
standardized test score alone. Rather than use high-stakes testing, schools can employ less costly strategies that are 
proven to support children's achievement, thus avoiding the social promotion/retention issue altogether. 



Smaller class sizes, especially in the primary grades, are frequently cited in the literature as promoting effective 
learning. "When class size goes down, learning goes up. It improves student achievement, particularly in the early 
grades and among students who are disadvantaged due to their socioeconomic background'* (DiMaria, 1999, p. 6). In 
small classes, students who may be a risk can easily be identified and therefore receive additional support throughout 
the year (DiMaria, 1 999). 



If children are grouped in un-graded or mixed-age classrooms groupings, classrooms where the age span is greater 
than one year, children have the opportunity to "progress according to their individual rate of learning and 
development without being compelled to meet age-related achievement expectations" (DiMaria, 1999, p. 6). When 
learning takes place in un-graded or mixed-aged classrooms, children will be able to advance to the next 
developmental level at their own pace without the restriction of grade-level labeling (DiMaria, 1999, p.7). 

We also advocate that schools should provide full-time kindergarten (NAECS, 2000) as well as offer alternative 
educational settings such as preschool, after school and summer school programs to assist those students that are 
lagging behind in certain academic areas (Thompson & Cunningham, 2000). These programs are critical for students 
who are economically disadvantaged and for whom English is a second language. 



When children are assessed, promotion decisions should be based on multiple assessments, not on a single test or a 
single administration of a test (Miller, 2000). And, when standardized testing is employed, the results are best used to 
identify problems so that swift remediation and curricular changes can be quickly implemented. (Miller, 2001). More 
preventative measures and less punishment in the form of grade retention should guide education policy decisions to 
give extra assistance to children found to be lagging behind (Thompson & Cunningham, 2000). 

Considering the Cost of Monetary and Human Capital 



Most politicians would argue that the cost of programs to improve teaching would be too expensive to implement. 
However, to get a clearer picture of the short and long term cost that will be incurred after the tougher retention policy 
takes effect, we must look at two factors, one a short term and the other a long term societal cost. In the short term, 
district operating budgets will be challenged by an increase in expenses when one considers that the cost of retaining a 
child for one year increases the educational cost for that child by eight percent (Foster, 1993). Longer-term costs are 
associated with the expense associated with the inter-generational poverty that will be perpetuated as retention 
encourages more students to drop out of school. According to a study completed by Grissom and Shepard (1989), an 
annual retention increase of five to seven percent will result in a cumulative increase of twenty percent to the present 
drop out rate. 



Proponents of retention might counter these two points by speculating that holding children back will improve their 
academic achievement with the reasoning that this improvement in academic skill will keep them in school. Yet, 
research paints a completely opposite picture with evidence that being overage in school plays a larger role in the 
decision to leave school than does academic achievement. Even when retained children do better academically, they 
drop out anyway. On average, the drop out rate is thirteen percent higher for over-aged children than the drop out rate 
for normal aged children (Grissom, & Shepard, 1989). Initially, retention advocates will boast that the drop out rate 
has been lowered as a result of the get tough accountability policy. But keep in mind that the first few graduating 
classes will have all the at-risk students removed. It is also important to look beyond high school drop out rates 
because many of the students who were retained will leave school before they enter high school. 

After an analysis of the CRCT results, it is apparent that the rate of retention will rise dramatically, precipitating a 
proportional increase in per student costs. To be fair, schools should only be held accountable for factors they can 
control, and, therefore, the decision to retain or promote a student should be made at the local level where socio- 
economic factors can be taken into account (Clotfelter & Ladd, 1996). In addition, the state of Georgia will also have 
to bear the burden of future societal costs if and when the student drops out of school. Is it not more cost effective to 
seek long-term lasting cures for poor student performance while students are in school rather than deal with the 
consequences later? 

The costs do not stop here. As we discussed earlier, school systems need to be aware of the legal ramifications of 
retention and the potential for litigation. Not only could parents bring litigation that schools did not provide adequate 
educational resources for their children to succeed (D. W. Albritten, Executive Director, Georgia Association of 
Educators, personal communication, November 24, 2001), they could also challenge unfair school policies. 



Recommendations 



Not only is there a preponderance of evidence that there is no academic benefit from retention, such practices also 
appear to be harmful to the social and emotional development of children. Since retention policies do not address the 
needs of under-prepared children, it is recommended that the limited resources of school systems be redirected toward 
the above listed alternative programs so that education is more responsive to what is best for children, not for 
institutions, politicians, or professionals. In the state of Georgia, several of these alternatives, such as full-time 
preschool and kindergarten, and smaller class sizes, have already been proposed or implemented. Instead of using 
high-stakes testing, Georgia's educational system needs to provide a prevention and remediation program during the 
school year rather than wait until after the student fails the criterion test. 

Moreover, education reform decisions need to be grounded in research rather than influenced by public demand for 
more stringent educational standards. If higher academic standards are the goal, then reform programs must speak to 
the reasons why student do poorly and offer meaningful assistance to improve learning outcomes. The educational 
community can no longer afford to ignore the consequences of policies and practices which disproportionately assign 
the burden of responsibility to the child rather than to the program, a reproach that places the child at risk of failure by 
generating apathy toward school and personal demoralization. 

Only when the stakeholders abandon the slogan that greater retention means more rigorous standards can schools 
begin to meet the pressing needs of children. By taking alternative paths away from retention, schools become more 
accountable to the holistic needs of the child. 
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Abstract 

Our analysis begins with an examination of the state of Georgia's rationale for the decision 
regarding social promotion that was based on the perceived views that teachers have on the issue. 

Research suggests, however, that teachers hold contradictory opinions concerning the use of 
standardized tests for high stakes decisions, such as promotion, and are not aware of the 
consequences most children suffer when they fail a grade. Following a discussion that challenges 
the claims of success in Chicago, Baltimore, and Texas, we explore the viability of choosing 
litigation as a strategy to stop the use of high stakes tests given the adverse impact they have on 
protected minorities. From a study of the thirty-nine poorest counties in rural Georgia, the 
relationships between poverty, race and the Georgia Criterion Referenced Competency Test 
Results suggest that these tests do have an enormously disparate impact on impoverished African 
American children. Because chances for educational attainment will be severely limited by this 
test, most African American children will be discouraged from achieving a high school diploma. 

As a way to put a face on the data, a case study of a young girl who would probably fail her 
grade in school if the law was enforced is presented followed by recommendations that argue for 
changes in education policy and teaching. Rather than mandate a ban on social promotion, the 
state of Georgia should pursue improvement of socio-economic conditions, education policy and 
pedagogy. 

For many, it is simple common sense to fail a child who does not pass the academic requirements for promotion to the 
next grade. For others, making children accountable for their academic performance is part of an overall strategy to 
raise the quality of education. Responding to the public and political call for educational reform and accountability, 
Georgia and Texas, Baltimore and Chicago along with many other school districts have decided to end the practice of 
social promotion in hopes of improving the quality of education (Eisner, 2000) 

Defining social promotion is somewhat difficult because the meaning behind the practice has become so infused with 
professional, political and academic agendas that it seems impossible to reach common ground. On one side are those 
who deride the practice as promoting a student from one grade to the next regardless of academic achievement; they 
claim that this is a policy that short-changes the child, teacher, school and society (DiMaria, 1999). On the other side 
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of the debate are those who point to the abundance of research that overwhelmingly suggests that keeping a child with 
his or her peer group is the best insurance for high school graduation (Darling-Hammond, 1998). Countering this 
plethora of research is the assumption that a high school diploma has little value if the student is simply passed along 
and given a degree without meeting specific criteria for graduation (Eisner, 2000). As evidence of the rift in the 
stakeholders' views, the two major teacher organizations cannot agree on the subject. The more conservative 
American Federation of Teachers (AFT) shudders at the thought that school districts ignore policies and laws that ban 
the practice of social promotion by stating that this rampant disregard creates a huge class of ill-prepared and 
unmotivated students (AFT, 2001). While conceding that social promotion without intervention is deleterious to a 
learner, the National Education Association (NEA) is on record as stating that retaining students is even more 
pernicious (NEA, 2001). 

Examining Georgia's Decision to End Social Promotion 

As a top priority for his education reform agenda. Governor Roy Barnes urged the legislature to end social promotion 
during the State of the State address in February 2001 . Governor Barnes proclaimed, "the time has come to end social 
promotion in our schools" (Barnes, 2001). Reasoning that social promotion is unfair to teachers. Governor Barnes 
charged the legislature with passing a bill that would require every student to pass an exit examination before being 
promoted to the next grade (Barnes, 2001). Using the rationale that schools and teachers are held accountable for 
competency, the Governor insisted that no student be promoted to the next grade level until proficiency in the subject 
matter has been assessed with a criterion based standardized test (Barnes, 2001). In Governor Barnes' words. 

Now, nobody wants to have to hold a child back in school. It is difficult for them to be separated 
from their peers. But if some children are still behind even after we have taken every step available to 
give them extra help-after school programs, alternative programs, special reading programs and so 
on-we owe it to them to make this difficult choice... . But mostly, we should do it in fairness to those 
students who are passing through our system today without learning what they need to know. By 
promoting a child who is not really ready, we say, 'It’s OK if you don’t learn.’ Well, I say, it is not 
okay (Barnes, 2001). 



On March 21, 2001, the legislature of Georgia complied with the wishes of the Governor by passing into law a bill 
which mandated that students in grades third, fifth, and eighth must pass a standardized examination to move up to the 
next grade, beginning with third graders in 2004. Children who enter third grade in 2003 would be required to pass a 
state reading test, while those matriculating the fifth grade in 2004 and eighth grade in 2005 would be required to pass 
both a state reading and mathematics exit examination (State Board of Education [SBOE], 2001). 

A second chance to pass the test is allowed if the child fails to pass the test on the first attempt. If the child should fail 
a second time, a grade placement committee is convened "composed of the principal or the principal's designee, the 
student’s parent or guardian, and the teacher of the subject of a test on which the student failed to perform 
satisfactorily’’ (SBOE, 2001). It is the initial charge of the grade placement committee to provide some sort of 
accelerated instruction to prepare the child for the third test. After three attempts, the official code directs the school 
to retain the student. At this point in the process, the parent(s) or legal guardian may appeal to the grade placement 
committee to permit the child to move up to the next grade level. Citing from Official Code 20-2-283, "The grade 
placement committee may decide in favor of a student’s promotion only if the committee concludes, using standards 
adopted by the local board of education, that if promoted and given accelerated instruction, the student is likely to 
perform at grade level. A student may not be promoted on the basis of the grade placement committee’s decision 
unless that decision is unanimous’’ (SBOE, 2001). 

Teachers Views on Social Promotion 

Governor Barnes’ contention that teachers are critical of social promotion may have some credibility when the 
research is examined. Tomchin and Impara (1992) published a study showing eighty-two percent of elementary 
school teachers believed that retention helps children prevent future failure and seventy percent thought that the threat 
of failure motivates children to succeed. A whopping ninety-eight percent stated that they would never rule out the 
decision to fail a child (Tomchin & Impara, 1992). DiMaria (1999), in a study of New York City teachers, found 
similar results. In this 1999 study, sixty percent of teachers felt that students should never be socially promoted with 
thirty percent reporting that the primary grades were the best times to retain in grade. 

Clearly, an overwhelming majority of teachers feel that social promotion frustrates children by burdening them with 
schoolwork that is too advanced for them to comprehend. Teachers believe that this burden is an imposition, one that 
makes teaching much more difficult because it forces them to deal with the under-prepared while trying to teach those 
who are prepared (Thompson & Cunningham, 2000). The prevailing view of teachers is that instruction is easier and 
more effective when variability of academic competence within the class is reduced (Foster, 1993). 

Both of the teacher assumptions about student and teacher frustration, however, are not borne out by the educational 
research. It may be that teachers are not aware of the preponderance of retention research as evidenced by their 
reliance on anecdotal accounts from colleagues. Anecdotal reports by teachers often suggest that children benefit from 
retention, yet because of the decision to retain, there is no opportunity to see how well the children might have 



progressed had they been promoted (NAECS, 2000). 



Mary Lee Smith wrote in Flunking Grades that teachers tend to access practical knowledge rather than formal 
knowledge. Practical knowledge is the sort of knowing that begins with personal experiences followed by future 
action based on these personal experiences. For example, one teacher remarked in Smith's study, "when my own son 
was retained, it was because he was too young for his age, and the next year he was a real leader in his class, and we 
never regretted that decision; and ever since then I have recommended that parents of young children in my class take 
the same step" (Smith, 1989, p. 133) 

Based on clinical interviews. Smith suggested that the educational philosophies of teachers fall into distinct categories 
and that these beliefs are directly related to their opinions on retention. One category Smith identified was designated 
Nativists, teachers who believe that the physiological maturation of ability develops overtime in stages. Predictably, 
Nativist teachers feel that children should not be exposed to developmental ly inappropriate instruction. If a teacher 
was not a Nativist, Smith found that teachers could be grouped in three additional ways. 1) Remediationists are 
teachers who are active instructional and resource managers, 2) Diagnostic Prescriptors believe that deficiencies, such 
as auditory memory and visual -motor problems, can be corrected when identified with specific instruction and 3) 
Interactionists, teachers who feel that successful teaching begins with the prior knowledge and interests that the child 
possesses (Smith, 1989). 

Smith found that the most likely to retain are Nativists because they are prone to see physical size and chronological 
age as reasons to hold children back. This is somewhat of a confounding finding given that Nativists' beliefs are 
congruent with some widely held theories of child development (Smith, 1989). While Nativists are more likely to 
retain students than the others, all teachers agreed in Smith's study that retention is beneficial both in the short as well 
as the long term. Through her interviews Smith recorded anecdotes such as the yearning to put the child at the top of 
the class and many claims to the effect that there are no stigmas attached to retention if the teacher and parents handle 
it well. Often Smith heard about the disasters that occurred when children are socially promoted and very few teachers 
named any negative effects of retention. All stated that it is best to err on the side of retention and if any harm was 
done, its effects are temporary. Perhaps the most disturbing finding Smith reported is that teachers often discounted 
the child's feelings of disappointment, failure and confusion or reluctantly acknowledged that if any emotional harm 
was done, the positives far outweigh the negatives. No one in Smith's research responded that social promotion was 
beneficial. Smith suggested that instructional efficiency pre-empts the child's best interest when she concluded, "The 
teacher is a self-interested theoretician. Though couched in the rhetoric of pupil benefits, her beliefs about retention 
are, perhaps unconsciously, conditioned by a wish for a more homogeneous and trouble free class" (Smith, 1989, p. 
149). 

Contradictions Between Research and Practice 

A preponderance of control-group studies, structured to measure the comparison between retained students and 
students recommended for retention but promoted anyway, come down clearly on the side of promotion. What these 
studies show is that students who are recommended for retention, but are nonetheless promoted to the next grade, end 
up doing just as well as or, in many cases, perform better academically than non-promoted peers (Foster, 1993; 
NAECS, 2000). 

While school performance is usually the focus of the debate, the most pernicious effect of retention is that the 
decision to fail a child usually results in dire social consequences. Children who have been retained demonstrate more 
social regression, display more behavior problems, suffer retention-related stress, and more frequently drop out of 
high school. (Grissom & Shepard, 1989; Frymier, 1997; NAECS, 2000). 



While it may seem unfair to some, keeping a child with his or her peer group is the best decision almost all of the 
time. When grade retention is used as a solution for poor performance, it is assumed that the problem resides in the 
child's learning ability. But, this is rarely the case (Darling-Hanimond, 1998; Frymier, 1997). 



Shedding light on why children do not do well in school, studies show that the reasons for poor performance usually 
stem from non-academic factors such as a seriously ill parent or the death of a sibling or maybe a parent lost his or her 
job last year (Frymier, 1997). In addition, many students who failed a grade had been in an accident or were seriously 
ill during the year (Frymier, 1997). Now that the 2000 census has been published, data show that many non-English 
speaking children have become part of our schools (U.S. Census, 2001). Research tells us that if English is not spoken 
at home, a child is twice as likely to be retained during his or her schooling (Foster, 1993). Another suggestive 
statistic is that over half of the students who were retained in grade came from a broken home where moving from 
town to town was a frequent experience (Frymier, 1997). Concomitant with these factors is what research has coined 
as "retention bias," a tendency to retain a higher proportion of males, those with small physical stature, poor children 
and minority students (Foster, 1993; Frymier, 1997; Miller, 2001). 

Because repeating a grade is a highly visible act, one that separates a student from his age peers, what is most 
disturbing about failing a child is what happens to them afterwards. Rather than accepting failure, children perceive 
the decision to repeat a grade as a punishment for something out of their control, a perception that discourages them 
from completing school (Foster, 1993). It is well documented that students who are held back do worse in the long 
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run compared to students who are promoted, in part because they give up on themselves as learners (Denton, 2001). 
Even small children perceive that failing a grade is a serious social stigma. Stigmatizing children lowers their self- 
esteem, a psychological albatross that often results in a teen pregnancy or drug and alcohol use later on (Darling- 
Hammond, 1998; Foster, 1993). A review of sixty-six studies conducted between 1990 and 1997 found that sixty-five 
of them showed retention to be ineffective or harmful (Denton, 2001). In another 1997 study of twenty-three risk 
factors for school failure, students who fail a grade have many more problems, in every risk area, than those students 
who were promoted to stay with their peers (Denton, 2001 ; Frymier, 1997; Owens & Magliaro, 1998). 

Standardized Tests and Consequences of Failure 

In preparation for the implementation of the legislation that bans the practice of social promotion, the state of Georgia 
has administered for the second year a Criterion Referenced Competency Test (CRCT), commonly referred to as the 
State Curriculum Test, as a way to determine how many children might be retained in grade. While some reading 
scores have improved on average, about twenty-five percent of fourth, sixth and eighth grade children flunked the test 
(Georgia Department of Education, 2001). Looking at the best results, eighteen percent of fourth grade students failed 
the reading component. In the worst performing category, forty-one percent of eighth grade children failed to achieve 
a passing grade in mathematics (Georgia Department of Education, 2001). What is alarming is that, beginning in 
2003, promotion to the next grade will be based on the results of this test, meaning that one in five, maybe more, 
students will be retained in grade. While there is some clamoring to revise the CRCT so that more students will pass, 
using a standardized test to make the critical decision to pass or fail a student has questionable validity (Salzer, 2001). 
The technical complexity of performance-based standardized tests, coupled with the fact that performance based tests 
are relatively new assessment and evaluation techniques, means that tests like the CRCT require constant revision in 
their early stages (Elmore, Abelman & Furhman, 1996). These changes are evidence that there are fundamental flaws 
in performance based tests, changes that result in improvement in some 2001 CRCT scores over 2000 CRCT scores. 

Even though the practice of standardized testing is hard pressed to show that it can produce real gains in student 
learning, state policymakers count on test revisions to improve scores. With standardized testing, researchers have 
found that scores will initially be low and then rise for several years before leveling off. This upward trend caused by 
the "saw tooth effect" is due to teacher-led test preparation rather than to student achievement (Miller, 2001). 
Predictably, policymakers tout the spike in test scores, usually within the second year of test introduction, as proof 
that accountability measures are working. 



While it is likely that the scores will get better because of this tinkering with the testing techniques and test 
preparation of students, there is no reliable evidence which suggests that performance-based tests will ever be 
perfected. Because the primary purpose of a standardized test is to gather data from a very large group of test takers as 
a way to evaluate if the overall curriculum needs to be improved, the CRCT should never be used to make a decision 
that affects an individual student (Miller, 2001). It is very important to note that a standardized test score does not 
reliably measure what an individual child actually knows because children are not consistent test takers. Even if the 
test was administered several times, the problem remains: snapshots cannot show a child's full range of capabilities 
(Kohn, 2000; Miller, 2001). 

Challenging the Claims of Success in Chicago, Baltimore, and Texas. 

With over fifty years of research showing that grade level retention provides no academic advantages to students, the 
practice of retention persists and is on the rise nationwide (Owings & Magliaro, 1998). The Consortium for Policy 
Research in Education (1990) reported that by the ninth grade, approximately fifty percent of all U.S. school students 
have been retained. If the goal of retention is to allow students more time to develop adequate academic skills so that 
they will be successful in subsequent years, why do the follow-up data on implemented programs throughout the 
United States show evidence to the contrary? 



Beginning in 1996, the Chicago public schools promoted only third, sixth, and eight graders who obtained the 
minimum score on the Iowa Test of Basic Skills. Initial studies purporting to show the success of Chicago's program 
revealed that students, especially those with the lowest prior test scores, showed impressive gains after a full year of 
intervention and intensive summer instruction. However, follow-up studies revealed that learning gains dissipated 
after three years resulting in an increased likelihood of school drop out (Thompson & Cunningham, 2000; Denton, 
2001; Holmes, 1989). In Baltimore, a similar story unfolded; a study conducted at Johns Hopkins University found 
that the performance of students retained during elementary school did improve modestly during the year they 
repeated and for several years thereafter (Denton, 2001). Again, follow-up studies revealed that initial gains faded, 
with sixty- five percent of the retained students dropping out of school as compared to eighteen percent of all other 
students. For students who were held back more than once, the drop out rate soared to ninety-four percent (Denton, 
2001). What unfolds as one examines the research is that retained children are, on average, worse off than those who 
are socially promoted (Holmes, 1989; Shepard et al., 1996). 



As for the Texas model, the study claiming success had serious methodological shortcomings that limit its validity 
(Denton, 2001). Texas researchers reported in a 1999 study that the performance of retained third graders improved 
over those students who failed the Texas Assessment of Academic Skills test but were promoted to the next grade. 
The validity of the test comes into question due to the large disparity between the number of students who were 
retained (400 students) and the number who were promoted (35,000 students). The 400 students, one percent of the 
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total, represented the extreme low end of the range of test scores, so any subsequent test scores had no place to go but 
up, a statistical phenomenon known as "regression to the mean." In addition to the test reporting flaws, Texas 
policymakers have also manipulated students to give the illusion of increased test scores. As reported by Haney 
(2000), school officials exclude poor test takers from the tenth grade TAAS by either retaining them in the ninth 
grade, classifying them as learning disabled, or encouraging them to leave school and pursue the GED. By employing 
these tactics, Texas schools can report apparent test score increases for the tenth grade students. 

A Snowball's Chance in Georgia: the viability of choosing litigation as a strategy to stop the use of 
high stakes tests to determine promotion or retention. 

Except for the mountains in the northern part of the state, it doesn’t snow very often in Georgia and, when it does, 
chances are that the frozen precipitation will not last very long. A snowball's chance in Georgia has the life 
expectancy of a fruit fly, about a day. The same analogy holds true for using the courts to overturn the legislature’s 
decision to use the CRCT to determine promotion and retention. While some cases across the nation have been won 
on the local level, almost all have been overturned at the appellate level, meaning that victory in the courts is short- 
lived. Appellate courts have overturned challenges based on two cases, United States v. Fordice in 1992 and 
Personnel Administrator v. Feeney in 1979. Basically, these two cases frarne the issue by deciding that "Placement 
testing, exit examinations, and achievement tests may be used to assist in the determination of classroom assignments 
and eligibility for graduation, provided that the test results are not a reflection of past racial segregation policies, the 
testing is accurate, and the results are open to public scrutiny" (Deskbook Encyclopedia of American School Law, 
2002, p. 489). Because the Equal Protection Clause of the Fourteenth Amendment forbids schools from engaging in 
intentional discrimination on the basis of race, color, national origin or sex , the first legal litmus test is whether or not 
a test perpetuates or preserves illegal discrimination. What the courts deemed important in Larry P. v. Riles in 1984 is 
that the State Department of Education had to foresee that the test would have a significant disproportionate impact by 
race. Second, the department of education has to have failed to show the validity of the test for minority children. 

Also, the test must cause a stigma and irreparable injury to the student. Moreover, Larry P. v. Riles demands proof 
that failing the test will result in effective educational opportunities for the child. (Heubert & Hauser, 1999). 



While the courts have recognized that high stakes tests have an adverse effect on minority children, they consistently 
reject the argument that these injuries are caused intentionally by the state. Instead, courts find that the state has a 
substantial governmental interest in education and that high stakes tests are a legitimate way to hold students 
accountable (Heubert & Hauser, 1999). Even when presented with clear accounts of racial bias, courts have refused to 
find that high stakes tests violate Title VI saying that they do not intentionally effect a particular race in an adverse 
manner (Heubert & Hauser, 1999; Deskbook Encyclopedia of American School Law, 2002). The Supreme Court 
forbids any practice that, while appearing to be a fair, perpetuates or promotes the effects of prior illegal segregation. 
This may mean that it is unlawful for any child who has attended an illegally segregated school at anytime in her/his 
schooling may not be subjected to a high stakes test designed for promotion or retention. It is rare today that a child 
has attended such a school, yet it opens the possibility that the courts could scrutinize a test more closely if a state or 
school district has had a recent history of segregation or intentional discrimination (Heubert & Hauser, 1999). 



The courts have almost uniformly dismissed claims of intentional discrimination and have steadfastly upheld that high 
stakes tests are rationally related to legitimate state interests (Deskbook Encyclopedia of American School Law, 

2002 ). 



While there is an abundance of research that shows that retention has deleterious effects, such as low self-esteem, 
negative attitudes toward school and a reduced chance at succeeding at school, the courts dismiss such reasoning as 
speculative. This view, that educational research is mere speculation, was evidenced through the case Erik v. by and 
through Catherine V.v. Causby North Carolina in 1997, a decision that upheld a school board’s decision to fail 
children based on a standardized test by rejecting the argument that students suffer irreparable harm when retained in 
grade because any potential harm is based solely on speculation. In stark contrast to the educational research, the 
court rationalized the situation completely oppositely saying that because retention gives a child more time in school 
to catch up, the state is doing its job by providing more resources to those who need them (Deskbook Encyclopedia of 
American School Law, 2002). 



o 

ERIC 



Texas courts also viewed retention as a part of a remediation benefit in the case, GI Forum v. Texas Education 
^Agency in 2000. In this case, retention was affirmed a part of a larger remediation process that provided those 
students who failed any portion of the exam with extra instruction intended to help them overcome their deficiencies. 
The court accepted the State's position that school accountability and mandated remediation helped to address the 
effects of prior discrimination in Texas because the exam provided the state with an objective way to assess student 
mastery of the skills and knowledge. When the issue of racial and cultural bias was raised, the court ruled that the 
exam was not fundamentally unfair to minority students because it measured what it claimed to measure and what 
was tested was taught. Because the test was aligned with the curriculum, the court decided that it was a valid test that 
met accepted standards (Deskbook Encyclopedia of American School Law, 2002). 



What the states are relying upon in court is a conservative legal viewpoint that legitimizes high stakes testing for 
promotion so long as the tests comply with generally accepted standards for its use. These generally accepted 
standards have two central principles: 1 .) a test score, like any other source of information about a student, is subject 
to error. Therefore, high stakes decisions like promotion should not be made automatically on the basis of a single test 



426 



score (Shepard &Smith 1987; Darling Hammond & Falk, 1995) and 2.) a student’s test score on a test should be used 
only in conjunction with other information sources in making such an important decisions as promotion to the next 
grade (Heubert & Hauser, 1999, p. 126). The state is clearly skating on thin ice here given that the generally accepted 
practice standards among psychometricians do not support the use of standardized tests as stand alone instruments to 
determine grade level promotion. Take for instance, it is generally accepted practice to supplement test scores with 
other assessment measures such as those performed by the teacher in the classroom (Heubert & Hauser, 1999; AERA, 
1985, 1998; Joint Committee on Testing Practices, 1988). Moreover, there is legal precedence that could disrupt the 
states’ position demonstrated by the decision made in the United States v. Fordice in 1992, a ruling that rejected the 
use of one test score for placement decisions (Heubert & Hauser, 1999). 

A legal strategy that offers a glimmer of hope rests on the concept of disproportionality. What must be proven is that 
grade retention is disproportionate among protected minority groups when compared to whites and that this 
disproportionality will decrease if equally reliable alternative assessments are used (Heubert & Hauser, 1999). It is 
well documented that grade retention is disproportionate among blacks/hispanics when compared to whites by a 
margin of 2: 1 (Heubert & Hauser, 1999). The data shows that by ages 10-1 1, ten percent more blacks and Hispanics 
are retained; by ages 15-17, forty to fifty percent more are retained and when students reach 15-17 years old fifty 
percent of blacks have fallen behind (Heubert & Hauser, 1999). There is a possibility that if the citizens of Georgia 
used disproportionality as a strategy, the state’s decision to use the CRCT could be ruptured when it is shown that 
those adversely affected are disproportionately protected minorities. Yet, any exuberance must be tempered with a 
good dose of healthy cynicism. 



In Texas, where the TAAS examination is used to determine promotion and retention, groups representing Texas 
minority students sued the state with the claim that the criterion referenced test discriminated against minority 
students in violation of the Due Process Clause of the 14th Amendment. What plaintiffs must show is that there is a 
preponderance of evidence that the policy of high stakes testing disproportionately has an adverse impact on a 
protected group, but this is not easy to determine. To prove adverse impact, proof must be presented via a study of the 
entire pool of test-takers that the success rate for members of a protected class is significantly lower than if a random 
sample was examined (Deskbook Encyclopedia of American School Law, 2002). Fortuitously for the children of rural 
declining Georgia, there is data available that has been compiled by the University of Georgia Department of Housing 
and Consumer Economics (2002) along with the State of Georgia’s Office of Education Accountability (2002) that 
meets the requirement for a study of the entire pool of test-takers showing that the success rate for members of a 
protected class is significantly lower than if a random sample was examined. 

Who Gets Hurt the Most: Relationships Between Poverty, Race and the Criterion Referenced 
Competency Test Results from Rural Declining Georgia. 

As a way to illuminate just how pernicious a law such as this will be to the poorest among us, we have conducted a 
study of school systems in the thirty-nine counties categorized as "declining rural counties" in Georgia, commonly 
referred to as the "black belt", so named because of the large number of African Americans who reside in them. 

Our methodology compared county by county demographic data compiled by the University of Georgia Department 
of Housing and Consumer Economics (University of Georgia Department of Housing and Consumer Economics, 
2002) with the State of Georgia’s Office of Education Accountability’s (State of Georgia’s Office of Education 
Accountability, 2002) statistics for each district. Because there is compelling evidence that family background is the 
primary determinate for school achievement (Shepard & Smith, 1989; Elmore, Abelman & Furhman, 1996; Clotfelter 
& Ladd, 1996), our study includes an analysis of eight socio-economic categories; 1) percentage of population that is 
African American, 2) per capita income, 3) children in poverty, 4) African Americans in poverty, 5) female headed 
families in poverty, 6) un-wed births, 7) percentage of population without a high school diploma, and 8) percentage of 
African Americans without a high school diploma. Given that family background is such an important predictor of 
success, it is critical to supplement the school lunch index, the common statistic used to determine poverty in schools, 
with multiple economic and cultural measures. 



When we first embarked on this study, two objectives were foremost: 1) compare the data gathered from these thirty- 
nine rural declining counties with statewide data; 2) present descriptive statistics that illuminate the relationship 
between CRCT scores and multiple socio-economic data. But, after we looked closer at the numbers, we discovered 
that in many of these counties, the school district data did not match up with countywide data. After comparing the 
county population demographics with the school systems data, it became apparent that many of these school systems 
have a discemable racial imbalance. Because this discovery suggests that race will matter in the decision to fail a 
child in Georgia, this research was expanded to include a discussion about the future of the rural African American 
community once the CRTC is implemented. 



These thirty-nine counties form a constellation of poverty that slashes through the southern region of the state of 
Georgia. Forming contiguous pockets of counties in rural decline, the constellation extends in a chain from the far 
southwest comer to the eastern part of the state. As a way to boost the clarity of the research, we have chosen to 
present this data through a geographic journey whose starting point begins in the most concentrated area of poverty in 
southwest Georgia. Traveling across the state, this study will explore those counties that make up the belt buckle, a 
band that traverses along the mid-section of the state from west to east, followed by a discussion of those counties that 
are located in the east. ^ ^ 



Southwest Rural Declining Counties 



These twelve counties are found huddled along the Alabama and Florida border in the farthest southwest comer of the 
State of Georgia framed by the Chattahooche River to the west, Albany, Georgia as the closest city to the east and 
Columbus, Georgia to the north. There are no major roads cutting through nor are there towns of any substantial 
population. While there may never be an occasion for many travelers to ever pay the folks here a visit, these twelve 
counties are home to 92,400 Georgians, of which, 14,080 are children in the public school system. 



Table 1 

Southwest Rural Declining Counties 
SES Data Compared to Georgia State SES Data 



SES Attributes 


Range in Southwest Rural Declining County Data 


State Data 


Population African American 


47.0% to 61.5% 


28.0% 


1999 Per Capita Income 


$16, 153 to $22,270 


$27,324 


1997 Children in Poverty 


27.2% to 47.4% 


21.8% 


1989 African American in Poverty 


33.7% to 53.0% 


30.3% 


1989 Female Headed Families in Poverty 


34.0% to 70.2% 


34.3% 


1999 Unwed Births 


42.9% to 65.9% 


36.6% 


No High School Diploma 


46.4% to 60.9% 


29.1% 


African American No High School Diploma 


56.6% to 70.6% 


41.4% 



As Table 1 shows, these twelve counties have two to three times more African American citizens than the rest of the 
state, most of whom live in poverty. Because school children here are likely to be poor and living in a household 
headed by an unwed mother who dropped out of school, the prospect for academic success is bleak. With up to 
seventy percent of the African American population lacking a high school degree, academic role models are hard to 
come by. When Table 2 is examined, it becomes apparent that all of these counties have reported similar socio- 
economic data. 



Table 2 

Southwest Georgia Counties in Rural Decline Socio-Economic Status 



County 


Pop. African 
American 


1999 Per 
Capita 
Income 


1997 

Children in 
Poverty 


1989 African 
American in 
Poverty 


1989 Female 
Headed Families 
in Poverty 


1999 

Unwed 

Births 


No High 
School 
Diploma 


African American 
No High School 
Diploma 


Baker 


50.4% 


$20,940 


35.7% 


35.0% 


41.1% 


41.7% 


46.4% 


56.6% 


Calhoun 


60.6% 


$21,646 


39.9% 


46.0% 


48.4% 


55.2% 


58.1% 


58.1% 


Clay 


60.5% 


$17,082 


46.9% 


50.6% 


52.3% 


57.8% 


48.6% 


63.4% 


Early 


48.1% 


$21,115 


42.2% 


51.5% 


61.7% 


57.0% 


45.9% 


62.4% 


Miller 


28.9% 


$22,270 


34.8% 


37.2% 


41.2% 


52.8% 


42.6% 


63.6% 


Mitchell 


47.9% 


$21,392 


35.8% 


45.3% 


53.8% 


57.0% 


45.1% 


63.5% 


Quitman 


46.9% 


$18,223 


47.4% 


52.3% 


70.2% 


63.6% 


50.5% 


68.8% 


Randolph 


59.5% 


$18,298 


43.3% 


53.0% 


55.8% 


57.3% 


50.7% 


64.6% 


Seminole 


34.7% 


$19,247 


39.3% 


47.5% 


48.5% 


49.7% 


48.6% 


60.9% 


Stewart 


61.5% 


$18,744 


38.0% 


45.2% 


56.4% 


48.6% 


60.9% 


70.6% 


Terrell 


60.7% 


$16,153 


38.7% 


42.4% 


51.1% 


65.9% 


47.6% 


65.3% 


Webster 


47.0% 


$20,728 


27.2% 


33.7% 


34.0% 


42.9% 


49.6% 


65.2% 


State 


28.0% 


$27,324 


21.8% 


30.3% 


34.3% 


36.6% 


29.1% 


41.4% 




A child attending school in these counties would have a one in three, at best a one in five, chance of passing the the 
third grade to the fourth grade once the CRCT decides his or her fate, a probability much worse than the rest of the 
state (Table 3). 

Passing on to the sixth grade will be even more difficult, given that your odds are about 50/50 that you will pass the 
CRCT, While the scores statewide are improving in the sixth grade CRCT, school’s scores are getting worse, ever 
widening the gap between rich and poor. If a child is so fortunate as to make it to the eighth grade in 2006, chances 
are better than even that he or she will not go to high school the next year because they failed the mathematics portion 
of the CRCT. As for comparing their school to the rest of the state (Tables 3 & 4), their school is in a free falling 
spiral, dropping significantly behind an abysmal statewide percentage of failing students. 

Table 3 
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Southwest Georgia Rural Declining Counties 



Percent Failing CRCT 4th Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


Southwest Counties 


32% 


31% 


50% 


Statewide 


26% 


26% 


38% 


Percent Change 


+23% 


+ 19% 


+32% 



Percent Failing CRCT 6^*^ Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


Southwest Counties 


37% 


47% 


46% 


Statewide 


24% 


36% 


31% 


Percent Change 


+54% 


+31% 


+48% 



Percent Failing CRCT 8^^ Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


Southwest Counties 


30% 


44% 


56% 


Statewide 


18% 


32% 


41% 


Percent Change 


+67% 


+38% 


+37% 



Table 4 

African American Southwest Georgia Rural Declining Counties 



Percent Failing CRCT 4^^ Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


African American Southwest Counties 


37% 


35% 


57% 


African American Statewide 


37% 


34% 


52% 


Percent Change 


0 


+3% 


+ 10 



Percent Failing CRCT 6^^ Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


African American Southwest Counties 


38% 


47% 


50% 


African American Statewide 


35% 


49% 


45% 


Percent Change 


+9% 


-4% 


+11% 



Percent Failing CRCT 8^*^ Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


African American Southwest Counties 


36% 


51% 


64% 


African American Statewide 


27% 


45% 


58% 


Percent Change 


+33% 


+13% 


+10% 



With the exception of one of the twelve counties, Webster County, the racial balance of the schools when compared to 
the general population is egregiously disproportional. Calhoun County's population is sixty percent African American, 
yet Calhoun County Schools have too few whites to report, meaning that forty percent of the white children in 
Calhoun County attend private schools or are home-schooled. Sixty percent of the fourth graders, forty-four percent 
of the sixth graders and sixty percent of the eighth graders in Calhoun County failed at least one CRCT content area 
test. 



Terrell County is the same story, only worse. Terrell is also sixty percent African American with no significant white 
representation in the schools. Having the lowest per capita income of around $16,000, Terrell County will face the 
fact that sixty-two percent of the fourth graders, sixty percent of the sixth graders and sixty-eight percent of the eighth 
graders will fail their respective grades. 




Quitman County, where forty-six percent of the population is African American, has no whites attending the one 
elementary public school there. In Quitman County, teachers and principals will face the daunting responsibility for 
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carrying out the failure sentence for eighty-one percent of the fourth grade class and sixty-five percent of the sixth 
grade. 

The same situation obtains in Randolph County, with ninety-three percent of their children on the free lunch program; 
teachers and principals there will be forced to fail sixty percent of the fourth, sixth and eighth grade students. While 
Clay County CRCT scores are not as low as the others, this all African American school system, with a per capita 
income of $17,000 and sixty-five percent of African Americans in the county without a high school degree, will fail 
twenty-nine percent in the fourth grade, fifty-five percent in the sixth grade and because forty-eight percent of the 
eighth grade did not meet the mathematics standards of the CRCT, they too, will fail. The remaining schools in the 
counties. Baker, Early, Miller, Mitchell, Seminole and Stewart, are also disproportionately African American when 
compared to the general population. Most of the schools in these counties are two-thirds African American with 
county data showing a range of one-third to one-half of the population as African American. 

Mid-State Rural Declining Counties 

Fourteen counties form a contiguous swath of land beginning in Talbot County, situated between Columbus and 
Macon Georgia, southward along Interstate 75 to the Florida border, where Clinch and Ware Counties envelope the 
great Okeefenokee Swamp. These mid-state counties are home for 168,276 Georgians, 28,854 of whom are children 
in the public schools. 

Table 5 paints a picture of economic and social crisis with data that shows per capita income well below the state 
average, resulting in significantly more children in poverty. As with the southwestern counties, school children in the 
mid-state counties are likely to have a parent who is a poor, unwed African American mother without a high school 
diploma. 



Table 5 

Mid-State Rural Declining Counties 
SES Data Compared to State 



SES Attributes 


Range in Midstate Rural Declining County Data 


State Data 


Pop. African American 


24.6% to 61.6% 


28% 


1999 Per Capita Income 


$15,585 to $23,202 


$27,324 


1997 Children in Poverty 


26.7% to 38.9% 


21.8% 


1989 African American in Poverty 


34.7% to 57.8% 


30.3% 


1989 Female Headed Families in Poverty 


37% to 64.7% 


34.3% 


1999 Unwed Births 


37.7% to 62.7% 


36.6% 


No High school Diploma 


39.4% to 53.8% 


29.1% 


African American No High School Diploma 


54% to 69.8% 


41.4% 



When each county is examined separately in Table 6, the relationship between race, poverty and educational 
attainment becomes clearer. Dooley, Macon, and Talbot, counties with the largest African American populations are 
the poorest; while Bleckley, Irwin, and Pulaski counties, with many fewer African Americans, are better off. These 
data suggest that this economic divide persists because of the lack of educational attainment among African 
Americans. When the column "African American No High School" is examined in Table 6, the data describe a 
population that has, for the most part, found it difficult to graduate from high school. Ten of these fourteen counties 
have anywhere from sixty to seventy percent of the African American population without a high school degree; the 
remaining four counties have fifty to sixty percent without a diploma. 

Table 6 

Mid-State Counties in Rural Decline Socio-Economic Status 



County 


Pop. 

African 

Am. 


1999 Per 
Capita 
Income 


1997 Children 
in Poverty 


1989 African 
Am. in Poverty 


1989 Female 
Headed Families in 
Poverty 


1999 

Unwed 

Births 


No High 
school 
Diploma 


African Am. No 
High School 
Diploma 


Bleckley 


24.6% 


$21,771 


26.7% 


42.0% 


37.0% 


41.8% 


39.7% 


66.7% 


Clinch 


29.5% 


$18,379 


32.9% 


40.9% 


53.6% 


49.5% 


53.8% 


62.6% 


Cook 


29.1% 


$18,276 


30.4% 


39.2% 


42.7% 


37.7% 


44.8% 


54.0% 


Dooley 


49.5% 


$18,690 


37.0% 


50.9% 


55.6% 


54.1% 


45.3% 


60.1% 


Irwin 


25.9% 


$20,832 


29.8% 


47.9% 


50.3% 


40.0% 


46.9% 


62.4% 


Lanier 


25.6% 


$17,675 


34.1% 


34.4% 


54.2% 


39.0% 


48.8% 


62.1% 


Macon 


59.5% 


$19,927 


37.1% 


39.8% 


49.3% 


62.7% 


46.3% 


57.3% 





















430 



Pulaski 


34.3% 


$23,202 


29.5% 


48.5% 


50.1% 


53.9% 


39.4% 


64.2% 


Taylor 


42.6% 


$18,774 


38.2% 


49.0% 


57.6% 


47.7% 


48.8% 


69.8% 


Telfair 


38.4% 


$18,477 


35.2% 


41.2% 


49.9% 


45.3% 


47.9% 


65.7% 


Turner 


41.0% 


$17,831 


38.6% 


57.8% 


64.7% 


53.5% 


44.7% 


64.3% 


Talbot 


61.6% 


$15,385 


34.1% 


34.7% 


46.4% 


48.8% 


43.8% 


58.2% 


Ware 


28.0% 


$19,738 


33.7% 


42.5% 


41.7% 


45.2% 


38.9% 


51.4% 


Wilcox 


36.2% 


$19,834 


38.9% 


56.2% 


61.4% 


51.2% 


47.2% 


69.6% 


State 


28.0% 


$27,324 


21.8% 


30.3% 


34.3% 


36.6% 


29.1% 


41.4% 



The socio-economic divide between Whites and African Americans in the declining rural counties of Georgia will 
surely be exacerbated through the implementation of the CRCT mandates. Because of retention in the third, fifth and 
eighth grades, African American children will be systematically encouraged eventually to drop out of school, 
resulting in the inability to command wages that might lift them out of poverty. Supporting the assumption that 
income is proportional to test scores. Tables 6 & 7 suggest that there is a relationship between CRCT test scores and 
the income earned by African Americans. When Mid-State African American CRCT scores are compared to 
statewide figures, the data show that poorer African Americans living in rural declining counties do worse than those 
African Americans who live in counties with higher income levels. Further, there is evidence that African American 
students who live in places where more of the African American population has earned a high school diploma do 
better on the CRCT than counties with less educational attainment. 

Table 7 

African American IMid-State Georgia Rural Declining Counties 



Percent Failing CRCT 4^^ Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


African American Mid-State Counties 


48% 


44% 


61% 


African American Statewide 


37% 


34% 


52% 


Percent Change 


+30% 


+29% 


+17% 



Percent Failing CRCT 6^^ Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


African American Mid-State Counties 


49% 


59% 


48% 


African American Statewide 


35% 


49% 


45% 


Percent Change 


+40% 


+20% 


+7% 



Percent Failing CRCT 8^^ Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


African American Mid-State Counties 


34% 


49% 


64% 


African American Statewide Counties 


27% 


45% 


58% 


Percent Change 


+26% 


+9% 


+10% 



The poverty to failure equation is repeated in Table 8 when the aggregate Mid-State CRCT scores show much lower 
results than the statewide data. 



Table 8 

Mid-State Rural Declining Counties 



Percent Failing CRCT 4^^ Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


Mid-State Counties 


36% 


33% 


45% 


Statewide 


26% 


26% 


38% 


Percent Change 


+36% 


+27% 


+18% 



Percent Failing CRCT 6^^ Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


Mid-State Counties 


33% 


46% 


34% 




Statewide 


24% 


36% 


31% 


Percent Change 


+37% 


+28% 


+10% 



Percent Failing CRCT 8^*^ Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


Mid-State Counties 


25% 


39% 


48% 


Statewide 


18% 


32% 


41% 


Percent Change 


+39% 


+22% 


+17% 



Eastern Georgia Rural Declining Counties 

Consider the Eastern Rural Declining Counties. These fourteen counties stretch vertically southward from counties 
that lie northwest of Augusta to rural areas southwest of Savannah. While these counties have very similar socio- 
economic data commensurate with very low CRCT test scores (Tables 9 and 10), some data are particularly 
noteworthy. Screven County's eighty percent mathematics failure in the eighth grade for African American students 
means that eight out of ten African Americans will not go to high school once the CRCT becomes the arbiter for 
promotion. Hancock County's statistics show that eighty percent of the children bom in 1999 in the county live in 
single parent households, meaning that the kindergarten class in 2004 would have eight out of ten children living with 
a single parent. 



Table 9 

Eastern Rural Declining Counties 



Percent Failing CRCT 4^*^ Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


Mid-State Counties 


38% 


38% 


49% 


Statewide 


26% 


26% 


38% 


Percent Change 


+46% 


+46% 


+29% 



Percent Failing CRCT 6^^ Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


Eastern Counties 


34% 


47% 


37% 


Statewide 


24% 


36% 


31% 


Percent Change 


+42% 


+31% 


+19% 



Percent Failing CRCT 8^^ Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


Eastern Counties 


27% 


46% 


51% 


Statewide 


18% 


32% 


41% 


Percent Change 


+50% 


+44% 


+24% 



Table 10 

African American Eastern Georgia Rural Declining Counties 



Percent Failing CRCT 4^^ Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


African American Eastern Counties 


47% 


45% 


61% 


African American Statewide 


37% 


34% 


52% 


Percent Change 


+27% 


+32% 


+ 17% 



Percent Failing CRCT 6^^ Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


African American Eastern Counties 


44% 


57% 


47% 


African American Statewide 


35% 


49% 


45% 
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Percent Change 



+25% +16% 



+4% 



Percent Failing CRCT 8^^ Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


African American Eastern Counties 


35% 


54% 


63% 


African American Statewide Counties 


27% 


45% 


58% 


Percent Change 


+30% 


+20% 


+9% 



When Tables 1 1 and 12 are compared to the southeastern and mid-state counties, there is a distinct outlier, Glascock 
County, a county with a small African American population of 8.3%, which is not consistent with the data from the 
other thirty-eight counties. Located in the center of a chain of five rural declining counties, Glascock County stands 
out as the only rural declining county that has SES and CRCT data better than, or comparable to, the State averages 
(Table 13). Because Glascock County's per capita income is in line with the other rural declining counties, the 
variable that confounds the repeated pattern of poverty and low CRCT scores is whiteness (Tables 14 & I 5). Not only 
will the vast majority of Glascock County students be promoted, about one third of the CRCT test takers actually 
exceeded the standards, a statistic not seen in any of the other thirty-eight county data. When the data from the other 
four counties in the chain are compared, Glascock County's relatively low unwed birth rate appears to reduce the 
number of children in poverty, suggesting that the "children in poverty" and "un-wed mothers" statistics may also be 
predictors for CRCT achievement. 



Glascock County's segregation from its neighbors leads to another assumption germane to this research, that the 
CRCT Test creates a new kind of discrimination - one that hides behind the appearance of fair testing to mask 
persistent inequalities in the quality of education that rural African American children receive in Georgia (McNeil, 
2000). Walter Haney, of Boston College's Center for the Study of Testing, warns that, "The consequences of 
standardized tests for Black and Hispanic students are clearly criminal from an educational point of view. It remains 
to be seen whether they are criminal under the United States Constitution" (McNeil, 2000, p. 231 ). 

Table 11 

Eastern Georgia Rural Declining Counties SES Data Compared to State 



SES Attributes 


Range in Eastern Rural Declining County Data 


State Data 


Pop. African American 


8.3% -77.8% 


28% 


1999 Per Capita Income 


$16,787-$2I,565 


$27,324 


1997 Children in Poverty 


22.3% -45.4% 


21.8% 


1989 African American in Poverty 


25.8% - 54.4% 


30.3% 


1989 Female Headed Families in Poverty 


29.4% - 64.2% 


34.3% 


1999 Unwed Births 


29.4% - 80.6% 


36.6% 


No High school Diploma 


38% -57.2% 


29.1% 


African American No High School Diploma 


54% - 80.6% 


41.4% 



Table 12 

Eastern Georgia Counties in Rural Decline Socio-Economic Status 



County 


Pop. 

African 

Am. 


1999 Per 
Capita 
Income 


1997 Children 
in Poverty 


1989 African 
Am. in 
Poverty 


1989 Female 
Headed Families in 
Poverty 


1999 

Unwed 

Births 


No High 
school 
Diploma 


African Am. No 
High School 
Diploma 


Emanuel 


33.3% 


$18,336 


36.6% 


46.1% 


43.5% 


54.6% 


47.4% 


65.7% 


Glascock 


8.3% 


$19,496 


22.3% 


29.6% 


29.4% 


27.3% 


49.7% 


80.6% 


Hancock 


77.8% 


$16,787 


37.4% 


33.8% 


49.6% 


80.6% 


50.5% 


54.8% 


Jefferson 


56.3% 


$17,673 


36.3% 


38.3% 


41.6% 


58.7% 


48% 


59.4% 


Jenkins 


40.5% 


$18,174 


37.4% 


46.8% 


53% 


57.6% 


50.1% 


71.2% 


Johnson 


37% 


$18,845 


36.3% 


38.3% 


41.6% 


58.7% 


48% 


59.4% 


Screven 


45.3% 


$19,181 


31.9% 


37.6% 


52.5% 


55.3% 


41.1% 


56.2% 


Taliaferro 


60.3% 


$17,383 


45.4% 


44.1% 


52.5% 


67.9% 


51.4% 


65.6% 


Tatnall 


31.4% 


$19,943 


34.6% 


42.7% 


50.3% 


44.3% 


42.6% 


56.9 


Treutlen 


33.1% 


$16,499 


37.5% 


47.3% 


48.2% 


47.1% 


47.3% 


56.4% 


Warren 


59.5% 


$17,664 


39.9% 


44% 


56% 


69.7% 


57.2% 


68% 


Wheeler 


33.2% 


$18,864 


37.9% 


54.4% 


64.2% 


32.5% 


43.3% 


54% 
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Wilkenson 


40.7% 


$19,614 


27.1% 


25.8% 


39.7% 


45.7% 


38% 


48.4% 


Wilkes 


43.1% 


$21,565 


29.4% 


37.1% 


40.5% 


50.8% 


43.4% 


62% 


State 


28.0% 


$27,324 


21.8% 


30.3% 


34.3% 


36.6% 


29.1% 


41.4% 



Table 13 

Glascock, Hancock, Jefferson, Taliaferro, and Warren Counties SES Data 



County 


Pop. 

African 

Am. 


1999 Per 
Capita 
Income 


1997 Children 
in Poverty 


1989 African 
Am. in 
Poverty 


1989 Female 
Headed Families in 
Poverty 


1999 

Unwed 

Births 


No High 
school 
Diploma 


African Am. No 
High School 
Diploma 


Glascock 


8.3% 


$19,496 


22.3% 


29.6% 


29.4% 


27.3% 


49.7% 


80.6% 


Hancock 


77.8% 


$16,787 


37.4% 


33.8% 


49.6% 


80.6% 


50.5% 


54.8% 


Jefferson 


56.3% 


$17,673 


36.3% 


38.3% 


41.6% 


58.7% 


48% 


59.4% 


Taliaferro 


60.3% 


$17,383 


45.4% 


44.1% 


52.5% 


67.9% 


51.4% 


65.6% 


Warren 


59.5% 


$17,664 


39.9% 


44% 


56% 


69.7% 


57.2% 


68% 



Table 14 

Glascock, Hancock, Jefferson, Taliaferro, and Warren Counties CRCT Data 



Percent Failing CRCT 4^^ Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


Glascock 


21% 


27% 


30% 


Hancock 


33% 


35% 


62% 


Jefferson 


46% 


43% 


56% 


Taliaferro 


60% 


57% 


65% 


Warren 


49% 


35% 


75% 



Percent Failing CRCT 6^** Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


Glascock 


23% 


34% 


20% 


Hancock 


40% 


51% 


41% 


Jefferson 


37% 


51% 


43% 


Taliaferro 


42% 


75% 


46% 


Warren 


60% 


67% 


66% 



Percent Failing CRCT 8^*^ Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


Glascock 


11% 


22% 


19% 


Hancock 


25% 


42% 


66% 


Jefferson 


35% 


53% 


63% 


Taliaferro 


47% 


59% 


76% 


Warren 


19% 


39% 


63% 



Table 15 

African American Glascock, Hancock, Jefferson, Taliaferro, and Warren Counties CRCT Data 



Percent Failing CRCT 4*^ Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


Glascock 


Too Few to Report 


Too Few to Report 


Too Few to Report 


Hancock 


32% 


34% 


62% 


Jefferson 


54% 


50% 


65% 


Taliaferro 


61% 


56% 


67% 


Warren 


46% 


34% 


74% 



Percent Failing CRCT 6^*^ Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


Glascock 


Too few to report 


Too few to report 


Too few to report 


Hancock 


40% 


51% 


41% 


Jefferson 


45% 


58% 


49% 


Taliaferro 


48% 


81% 


48% 


Warren 


60% 


66% 


65% 



Percent Failing CRCT 8^^ Grade 



Content Area 


Reading 


English/ Language Arts 


Mathematics 


Glascock 


Too few to report 


Too few to report 


Too few to report 


Hancock 


26% 


43% 


67% 


Jefferson 


36% 


56% 


67% 


Taliaferro 


50% 


63% 


81% 


Warren 


19% 


39% 


63% 



Giving Up on Going to High School 

Overwhelmingly, African Americans in Rural Declining Counties are at much greater risk of failing the fourth, sixth 
and eighth grade CRCT than African Americans who live in cities, suburbs or rural growth counties. Yet, those most 
at risk are eighth graders who attend all-Black schools in the rural declining counties. Because they failed one or more 
of the content area tests, chances are that most of the eighth graders in these ten counties will not go on to high school. 
Clearly, the worst performing category was mathematics with only forty-four percent eligible to move up to the ninth 
grade. If the law were effective today, Taliaferro County would send four students to high school leaving thirteen 
behind, Talbot County would send only eleven, holding back forty-four. Calhoun fails thirty-three of their fifty-seven 
eighth graders, Clay County retains half of their thirty-two children. In the larger counties, Hancock retains eighty of 
one hundred twenty-one as Terrell County keeps ninety-one of their one hundred thirty-three eighth graders in middle 
school. Ten of the thirty-nine Rural Declining Counties fall into the 'all-Black' school system category with six of 
these, Calhoun, Quitman, Clay, Randolph, Stewart and Terrell, being located in the Southwest section of the State. 

The others, Taliaferro, Hancock and Warren surround all white Glascock County in the east with Talbot County being 
the lone all-Black school district in the Mid-State region. 



Because each of these counties has its own school district, these schools are not considered illegally segregated. 
Drawing school district lines by county does not, superficially at least, appear to be gerrymandering given that each 
school district corresponds to an established county. Yet, segregation is, nonetheless, the result and the children in 
these schools suffer all the doisadvantages of a segregated education. 



Future Consequences 

Being the most impoverished counties in the state, these Rural Declining school systems faced formidable challenges 
before the legislation to end social promotion was passed. With an average of twenty-five percent of their populations 
under the age of seventeen, these counties have large numbers of children who need enormous resources to overcome 
obstacles to academic success. What makes the "Declining Rural Counties" of Georgia's plight unique is that the 
children who attend schools in these counties will be denied de facto their property rights to a public education when 
they drop out of school as a result of the practice of grade retention. From the data presented here, most of these 
counties already have very high drop out rates. What percentage will the drop out rate reach when thirty-five to fifty 
percent of all fourth, sixth and eighth graders will be retained in grade? The question that ought to be asked is "Is this 
legislation really intended to improve education or is it a strategy to reduce the State's financial obligation to the rural 
poor?" It is clear that failing masses of poor children will not improve pedagogy because punishing children with 
retention does not change teaching. What we do know is that the association between retention and dropping out is 
noted consistently throughout educational research. Without a doubt, flunking children increases the risk of dropping 
out of school (Frymier, 1997). Because these thirty-nine counties are very poor, and the tax base available for public 
schools is small, the State of Georgia compensates for this revenue deficiency by making exceptionally large 
contributions to these counties. Thus, while not stated as policy, it cannot be ignored that the CRCT will most likely 
save the State a considerable amount of money by reducing the number of students in schools in these counties. 



While we dispute claims that the CRCT is a valid instrument to determine if a child should be retained in grade, we do 
not dispute that the CRCT is ironically a very reliable measure of economic resources (Kohn, 2001 ). As legislators 
extol the virtues of achieving academic excellence by using a "fair" test, like the CRCT, to determine if a child passes 
or fails, some wonder if the real agenda is a subtle form of class warfare intended to institutionalize intergenerational 
immobility and social stratification (Ohanian, 1999), a kind of violence that leaves behind the children with the least 
resources (Spring, 2000). Given the correspondence between the economic system and the role that the institution of 
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education plays in perpetuating the class stratification of society (Bowles & Gintis, 1976), this legislation guarantees 
that the social reproduction of the society in rural Georgia will be preserved. The claim that flunking these children is 
for their own good is unconscionable, considering the dire social and economic punishments that will be imposed 
upon them. Ohanian derided such accountability measures "as cynical as handing out menus to homeless people in the 
name of eradicating hunger" (Ohanian, 1999, p. 31). 

Chelsea's Story: Putting a Face on the Data 

We first met Chelsea at the garage where we have our car fixed; it was about 3:30 p.m. on a sweltering spring day in 
rural South Georgia. Her great-grandmother had just picked her up from school to bring her to the garage to finish out 
the workday. Great-grandmother sat down at her desk where she worked as the receptionist and bookkeeper for the 
family owned garage. As I waited for my car to be repaired, we both asked the light brown skinned six-year-old with 
big brown eyes and braided black hair if she enjoyed her day at school. Chelsea smiled broadly, "Yes!" she 
exuberantly replied as she showed us the cover of a book she brought home. From all appearances, Chelsea was a 
healthy, well-loved first grader. But, this has not always been the case for Chelsea. Just recently, her great- 
grandmother took legal measures to have Chelsea taken from her mother after a man who was living in the home 
broke Chelsea's arm while trying to spank her. After he broke Chelsea's arm, the boyfriend badly burned her baby 
brother because he was crying too loudly. This abuse by one of Chelsea's mother's boyfriends was one of many 
horrors that this beautiful little girl had to endure over the course of her short life. As great-grandmother tells it, her 
grandson had a short affair with Chelsea's mother, an affair that brought Chelsea into the world. After her grandson 
left Chelsea's mother, great-grandmother felt obligated to support Chelsea in any way that she could. The offer to give 
financial support and emotional support was a selfless act of love given that great-grandmothers' extended family is 
barely making ends meet, in addition to the financial hardship, it was a painful situation for great-grandmother to 
witness the cruelty inflicted on Chelsea. Chelsea's mother was involved with too many men and had too many 
substance addictions to care for her children. While great-grandmother reported to the social service system many 
times that Chelsea was being neglected and abused, Chelsea remained with her mother, it took an act of violence 
against the children for the system to intervene on Chelsea's behalf Even after the occurrence of such crimes 
committed against helpless children, Chelsea must still comply with a visitation plan that includes overnight stays 
with her mother. 

Behind in School. After a conversation with great-grandmother later in the Spring, we became aware that Chelsea did 
not do very well academically during the year. As a result of her poor academic performance, due to excessive 
absences in the beginning of the year, Chelsea's teacher recommended that she repeat the grade. Already retained once 
in kindergarten, Chelsea’s great-grandmother was reluctant to hold her back again, great-grandmother felt that by 
Chelsea now being in a loving and secure home, the prospect that she will catch up to her peers is promising. 

Chelsea’s teacher is basing her recommendation to retain on the failing grades that Chelsea received during the school 
year. Yet, if the Iowa Test of Basic Skills and a psychological assessment conducted at the behest of the Department 
of Family Services are considered in the decision, it is clear that Chelsea's poor grades were the result of poor 
attendance rather than cognitive ability. While Chelsea was absent for most class time when she lived with her 
mother, her scores were not as low on the ITBS as one might expect, and in some linguistic areas, her scores were 
high. As a result of high scores in certain verbal skills, Chelsea scored a composite word analysis in the sixty-fifth 
percentile. Chelsea's overall math abilities were average, but her composite math score was low because of a very low 
score on the computation section of the test. 

In support of the decision to promote, Chelsea scored at a grade level of 2.2 in reading and 2.0 in math on the 
Kaufman Test of Educational Achievement in a low stress testing environment. The Kaufman Test showed that 
Chelsea is beginning to read independently and can add and subtract simple numbers. As an indicator of her mental 
ability, the Weschler Intelligence test showed that Chelsea is a child of average intelligence. The psychologist who 
observed Chelsea noted that she was a cheerful child with no signs of depression. The drawings she drew for the 
psychologist showed a loving family situated around a sturdy tree and a solid house. The psychologist concluded that 
Chelsea was now in a loving family that could be trusted for support. 



The assessment went on the note that Chelsea is not a behavioral problem in school, she has many friends and is 
openly curious about new things. She is articulate, attentive, with good concentration and perseverance. Chelsea does 
not have any violent tendencies and, it is worth noting, that in light of her own physical abuse, she is quoted as saying 
that it is wrong to hurt anyone else. Having a birthday in October of 1 993 makes Chelsea one of the oldest in her first 
grade class. If her great-grandmother had not insisted that Chelsea be promoted, holding her back again may have 
compounded her risk for dropping out of school later in life. Chelsea is progressing very well in the second grade and 
her chances for success are bright. Yet, another story could have unfolded if a standardized test was the arbiter for her 
future. 

Preventing Failure with Changes in Education Policy 

The fact that many school systems are overwhelmed by the increased number of under prepared students and use 
social promotion as a necessity, the public backlash should not be directed to testing and retention as the answers. 
Retention policies should be highly suspect given the lack of demonstrated effectiveness and prevalent bias against 
certain groups of children. Focusing public attention on standardized testing directs attention away from input issues 
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that can substantially effect quality education and places the blame instead on students and teachers. Because the 
negative effects of failure on children's achievement, motivation, self-concept, and graduation rates are so well 
known, one of the most important decisions in a young person's life should not be based on the outcome of a 
standardized test score alone. Rather than use high-stakes testing, schools can employ less costly strategies that are 
proven to support children's achievement, thus avoiding the social promotion/retention issue altogether. 

Smaller class sizes, especially in the primary grades, are frequently cited in the literature as promoting effective 
learning. "When class size goes down, learning goes up. It improves student achievement, particularly in the early 
grades and among students who are disadvantaged due to their socioeconomic background" (DiMaria, 1999, p. 6). In 
small classes, students who may be a risk can easily be identified and therefore receive additional support throughout 
the year (DiMaria, 1999). 

If children are grouped in un -graded or mixed-age classrooms groupings, classrooms where the age span is greater 
than one year, children have the opportunity to "progress according to their individual rate of learning and 
development without being compelled to meet age-related achievement expectations" (DiMaria, 1999, p. 6). When 
learning takes place in un-graded or mixed-aged classrooms, children will be able to advance to the next 
developmental level at their own pace without the restriction of grade-level labeling (DiMaria, 1999, p.7). 



We also advocate that schools should provide full-time kindergarten (NAECS, 2000) as well as offer alternative 
educational settings such as preschool, after school and summer school programs to assist those students that are 
lagging behind in certain academic areas (Thompson & Cunningham, 2000). These programs are critical for students 
who are economically disadvantaged and for whom English is a second language. 



When children are assessed, promotion decisions should be based on multiple assessments, not on a single test or a 
single administration of a test (Miller, 2000). And, when standardized testing is employed, the results are best used to 
identify problems so that swift remediation and curricular changes can be quickly implemented. (Miller, 2001). More 
preventative measures and less punishment in the form of grade retention should guide education policy decisions to 
give extra assistance to children found to be lagging behind (Thompson & Cunningham, 2000). 

Considering the Cost of Monetary and Human Capital 

Most politicians would argue that the cost of programs to improve teaching would be too expensive to implement. 
However, to get a clearer picture of the short and long term cost that will be incurred after the tougher retention policy 
takes effect, we must look at two factors, one a short term and the other a long term societal cost. In the short term, 
district operating budgets will be challenged by an increase in expenses when one considers that the cost of retaining a 
child for one year increases the educational cost for that child by eight percent (Foster, 1 993). Longer-term costs are 
associated with the expense associated with the inter-generational poverty that will be perpetuated as retention 
encourages more students to drop out of school. According to a study completed by Grissom and Shepard (1989), an 
annual retention increase of five to seven percent will result in a cumulative increase of twenty percent to the present 
drop out rate. 

Proponents of retention might counter these two points by speculating that holding children back will improve their 
academic achievement with the reasoning that this improvement in academic skill will keep them in school. Yet, 
research paints a completely opposite picture with evidence that being overage in school plays a larger role in the 
decision to leave school than does academic achievement. Even when retained children do better academically, they 
drop out anyway. On average, the drop out rate is thirteen percent higher for over-aged children than the drop out rate 
for normal aged children (Grissom, & Shepard, 1989). Initially, retention advocates will boast that the drop out rate 
has been lowered as a result of the get tough accountability policy. But keep in mind that the first few graduating 
classes will have all the at-risk students removed. It is also important to look beyond high school drop out rates 
because many of the students who were retained will leave school before they enter high school. 



After an analysis of the CRCT results, it is apparent that the rate of retention will rise dramatically, precipitating a 
proportional increase in per student costs. To be fair, schools should only be held accountable for factors they can 
control, and, therefore, the decision to retain or promote a student should be made at the local level where socio- 
economic factors can be taken into account (Clotfelter & Ladd, 1 996). In addition, the state of Georgia will also have 
to bear the burden of future societal costs if and when the student drops out of school. Is it not more cost effective to 
seek long-term lasting cures for poor student performance while students are in school rather than deal with the 
consequences later? 



The costs do not stop here. As we discussed earlier, school systems need to be aware of the legal ramifications of 
retention and the potential for litigation. Not only could parents bring litigation that schools did not provide adequate 
educational resources for their children to succeed (D. W. Albritten, Executive Director, Georgia Association of 
Educators, personal communication, November 24, 2001), they could also challenge unfair school policies. 



Recommendations 
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Not only is there a preponderance of evidence that there is no academic benefit from retention, such practices also 
appear to be harmful to the social and emotional development of children. Since retention policies do not address the 
needs of under-prepared children, it is recommended that the limited resources of school systems be redirected toward 
the above listed alternative programs so that education is more responsive to what is best for children, not for 
institutions, politicians, or professionals. In the state of Georgia, several of these alternatives, such as full-time 
preschool and kindergarten, and smaller class sizes, have already been proposed or implemented. Instead of using 
high -stakes testing, Georgia's educational system needs to provide a prevention and remediation program during the 
school year rather than wait until after the student fails the criterion test. 

Moreover, education reform decisions need to be grounded in research rather than influenced by public demand for 
more stringent educational standards. If higher academic standards are the goal, then reform programs must speak to 
the reasons why student do poorly and offer meaningful assistance to improve learning outcomes. The educational 
community can no longer afford to ignore the consequences of policies and practices which disproportionately assign 
the burden of responsibility to the child rather than to the program, a reproach that places the child at risk of failure by 
generating apathy toward school and personal demoralization. 

Only when the stakeholders abandon the slogan that greater retention means more rigorous standards can schools 
begin to meet the pressing needs of children. By taking alternative paths away from retention, schools become more 
accountable to the holistic needs of the child. 
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Teacher testing was inaugurated in Massachusetts in 1998 and a 59% failure rate among test- 
takers led to public shaming of the teacher candidates and their colleges and universities in the 
media. Within a two-year time period, low-performing teacher education programs in 
Massachusetts initiated a wide range of test preparatory activities which led to a dramatic 
increase in their students' pass rates. The authors separate colleges and universities into three 
categories and examine their differentiated responses to teacher testing. Their finding that 
institutions of higher education have responded effectively to teacher testing does not preclude 
critique of teacher testing as currently practiced in Massachusetts. 



Teacher testing has emerged as one of the most widely disseminated educational practices related to the improvement 
of teacher quality in the United States in the last twenty years. What was once a state concern, however, has now 
become federal. With the reauthorization of Title II of the Higher Education Act (Public Law 105-244) in 1998, states 
are now required to report to the United States Department of Education on how well their teacher education program 
completers fared on teacher tests. This legislation, in effect, federalized teacher testing. 

Under this legislation, colleges and universities with failing teacher education programs stand to lose federal funding 
for professional development programs, research and student financial aid. Institutions of higher education (IHEs) 
with low passing rates risk the humiliation of being publicly designated as "low-performing" by their states (United 
States Department of Education, 2000). Teacher education programs whose students fail to meet minimum pass rates 
face sanctions and, ultimately, closure by state departments of education (Massachusetts Department of Education, 
1998). 

A variety of concerns have been raised about teacher testing as a new federal policy in the United States (National 
Research Council, 2001). Some critics have worried that the tests filter competent teachers, especially minorities, out 
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of the profession (Melnick and Pullin, 2000); others question whether the tests measure the most critical attributes of 
teachers (Flippo and Riccards, 2001); and others have raised technical questions about the tests (Haney, Fowler and 
Wheelock, 1999; Ludlow, 2001). Many of these concerns are legitimate and require further clarification and debate 
among teachers, policymakers, and the public at large. 

Regardless of the concerns that have been raised, however, colleges and universities with teacher education programs 
are currently held '’accountable" for the results of their candidates on the tests. Institutions across the country have 
considered a variety of options for responding to this new testing regimen, of which three are most salient. They can 
(i) change their teacher education curricula to align them with the test; (ii) restrict their applicant pool to exclude 
applicants who they believe might not be able to pass the test; or (iii) develop test preparation workshops to address 
areas of students' academic weakness. 



Thus far, we have little data on transformations in teacher education curricula or restrictions on applicants. In their 
research at five colleges and universities in Massachusetts conducted during the first year after the test was 
implemented, our colleagues Marilyn Cochran-Smith and Curt Dudley-Marling found little evidence of changes in 
teacher education coursework relevant to the test (2001). Furthermore, even though there has been some speculation 
that there have been restrictions in the applicant pool, we do not have any quantitative evidence to test this hypothesis 
at this point in time. 



We take it for granted that teacher testing is now an established part of the American educational landscape, and we 
are skeptical that any social movements or political coalitions will arise which will have sufficient power to terminate 
teacher testing. The testing movement has been firmly embraced by the current administration in Washington, DC, 
and while there are some opponents, they do not seem to enjoy broad public support. Given that these tests are here to 
stay, we seek to pose and answer an educational question: what is it that teacher education programs are doing to 
ensure that their students* pass rates meet state standards? 

Three Core Questions 

To answer this question, we gathered data from our home state, the Commonwealth of Massachusetts. We asked 
faculty and staff at all 59 different teacher preparatory institutions to answer three essential questions. These 
institutions ranged from small private colleges graduating no more than a dozen teacher candidates each year to large 
state colleges and universities with hundreds of future teachers completing their course work annually. 

First, we asked what ejforts are currently underway at your institution to prepare students to take the Massachusetts 
Test for Educator Licensure (MTEL)1 For example, has your institution (a) recommended that students not take all 
three tests on the same day, (b) advised students to take tests earlier in their undergraduate programs to allow repeat 
opportunities to take the test, (c) offered "test taking skills" sessions and, if so, what is the purpose, format, and 
content of those sessions, (d) coordinated with arts and sciences departments to ensure coverage of subject matter 
tests, or (e) changed curricula to add or drop subject coverage? 

Second, we asked what mechanisms does your institution have for keeping track of and analyzing student results? 
Does your institution create data files containing student test results? If so, who maintains the files and conducts 
analyses upon them? What kinds of analyses are performed with the data? For example, are the test results statistically 
analyzed by degree level or program? Or, are test scores related to SATs and GPAs? What kinds of analyses of 
students who are potentially at-risk have been conducted — and have those analyses informed intervention strategies? 
What curricular decisions have resulted from analyses of the test scores? 

Third, we asked what improvements in MTEL pass rates can be directly linked to curricular changes and test 
preparation changes? Specifically, is there any evidence to suggest that these changes or test preparation sessions 
improve the chances of passing the tests on the first time (or on subsequent test-taking)? What test areas have shown 
improvements in pass rates? How have pass rates, in general, changed over time? 

Key Data Sources 

These questions were sent to each Title II Coordinator at the 59 teacher preparation institutions in the 
Commonwealth. A follow-up request for participation was sent two weeks after the initial mailing and another was 
sent two weeks after that. Some of their email responses prompted follow-up questions from us. In addition, some 
Title II Coordinators participated in phone or face-to-face interviews. Thirty-nine institutions ultimately provided 
descriptions of how they prepare students for the tests (Note 1). For the 1999-2000 Title II reporting period, these 39 
institutions served 94% of the program completers tested in the state. 

Before answering our first query about efforts currently underway to prepare teacher candidates for the MTEL, a 
reminder is warranted about the historical origins of teacher testing in Massachusetts. The first administration of the 
teacher test in 1998 resulted in a 59% failure rate and generated national news when the speaker of the House of 
Representatives condemned the failed candidates as "idiots" (Pressley, 1998). Colleges and universities with high 
failure rates suffered public condemnation. Chastened deans, provosts, and presidents found themselves in the 



unfamiliar role of apologists for their teacher preparatory programs. Critics within schools of education condemned 
the teacher tests as a set-up insofar as the candidates knew little about the content on the tests and were told 
consistently (until shortly before the tests) that the results would not count. Professors, staff, and students in schools 
of education took careful note of how their own institutions sized up in relationship to their competitors. 

In this climate of intensive external scrutiny and criticism and self-reflection and analysis, IHEs were compelled to 
implement any strategy that appeared to not just improve their curriculums but also to show the public they were 
responding to this perceived failure in their programs to prepare teacher candidates. No guidance, however, was 
provided by the Massachusetts Department of Education about what curricular components should be changed, added, 
or eliminated. Nor were any test results available from the test contractor (National Evaluations Systems: NES) that 
provided useful diagnostic information to students and programs about specific test content deficiencies (Note 2). As 
a result, programs were left to their own devices to create survival strategies. 

These strategies were largely uncoordinated and were implemented at the discretion of education deans, department 
chairs, and program coordinators. A number of institutions hired psychometricians to help them build data files to 
make sense of the test results. Another strategy consisted of the immediate implementation of test preparation sessions 
taught by testing faculty and academic development centers. Over time, word of these efforts spread and colleges and 
universities began sharing their strategies with one another. With the pressure of Title II reporting of teacher 
candidate test results, the need for a more systematic survey and appraisal of outcomes became apparent to all parties. 

As a result of the expressed interest from colleges and universities to the Massachusetts Department of Education, the 
Department conducted a survey in the fall of 2001. The survey solicited data from IHEs relevant to new test 
preparation programs and addressed issues of intended audiences and expense. The survey did not inquire into the 
substance of the test preparation programs nor did it seek to gauge their successes. 

First Question: Levels of Institutional Investment 

Based on the extensive and enthusiastic responses we received we have organized our survey results around the theme 
of institutional investment. This theme draws on the work of Cochran-Smith and Dudley-Marling, who noted that "the 
speed and degree with which institutions shifted resources to test preparation was linked to the degree of urgency and 
crisis that participants perceived at their institutions" (2001). Our objective was to document the extent and types of 
resource shifts and their real and apparent efficacy. We identified three levels of institutional investment; those 
heavily, moderately, or minimally invested in shifting institutional resources to immediately address the need the raise 
teacher test pass rates. 

We describe institutions of higher education with high numbers of candidates who failed on the first administration of 
the test as "besieged." In general, fewer than forty percent of teacher candidates at these colleges and universities 
passed the first three administrations of the teacher tests in 1998. These besieged institutions have as an aggregate 
responded with an extensive, heavy investment to reduce their high initial failure rates. Of our responding institutions, 
20 or 51% fall into this category. 

Besieged institutions have developed a host of strategies to help their teacher candidates pass the MTEL. 
Administrators — usually deans — at these colleges and universities have hired consultants to teach staff how to 
conduct their preparatory workshops or they have hired consultants to conduct the workshops directly for future test- 
takers. Staff have created in-house test preparation workshops which extend across weeks and even months. Faculty 
and staff have designed and taught new one-credit courses to help students prepare for the test over the course of an 
entire semester. Administrators hired statisticians to create sophisticated longitudinal data bases for the purpose of 
tracking individual student performances. In addition, these statisticians built models to identify program strengths 
and weaknesses. 



At many besieged institutions, we observed a university-wide response to the crisis of low test scores. Especially 
noteworthy in this regard is the manner in which collaboration between teacher education and academic content 
faculty was catalyzed by a mutual desire to raise the scores. At one IHE, the committee leading the drive to prepare 
students for the test included the Vice President for Academic Affairs (who chaired the committee), the Dean of the 
School of Arts and Sciences, the Dean of the School of Education, the Associate Vice President for Student Services, 
the Director of Academic Advisement, and the Title II Coordinator (who was a professor of education). 



At another heavily invested institution an all-out effort was mounted to raise teacher test scores. Teacher candidates 
were offered a variety of test preparatory workshops, ranging from two to twenty-four hours in duration. These 
workshops included not only the communication and literacy segments of the test, but also content area sections, such 
as history, mathematics, or chemistry. Faculty received professional development assistance in redesigning their 
course curricula to include test-taking skills; individual tutors were hired to help struggling teacher candidates; and 
the results of students' writing outcomes on the MTEL were correlated with changes in expository writing classes as 
part of extensive program assessment. Some of the test preparation classes were offered for academic credit. 
Academic advising emphasized the importance of the tests and informed students of multiple opportunities to prepare 
for them. 
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Faculty familiar with the design of the MTEL have served as guest lecturers in classes taken by prospective teachers; 
IHEs have hired additional faculty to teach writing skills and to serve as part-time academic advisors; and the linkage 
between state curriculum frameworks and the teacher test has been taught explicitly in methods classes. Some 
institutions have developed entire new courses to help their students pass the test, and some have increased 
distributional requirements with the goal of helping the students to improve their English language capabilities. 

Many institutions have concentrated on the early identification of at-risk students, often during the freshman year. For 
example, one institution requires its freshmen in their first undergraduate course to take the PRAXIS test so that 
faculty in the teacher education program can immediately identify a students' strengths and weaknesses in regards to 
communication and literacy. In this instance, PRAXIS serves as a kind of pre-test for the teacher candidates. 

One of the more surprising innovations developed by these institutions concerns the use of middle and high school 
textbooks as test preparatory materials. As discussed by Ludlow (2002) at a recent regional workshop for teacher 
educators, a teacher candidate might be better served by studying a high school history or biology textbook in depth 
than to take specialized courses in social history or evolution that contain large bodies of information not measured on 
the test. (Whether the focus on high school texts diminishes teacher candidates' awareness of recent debates in history 
or discoveries in genetics is an open question.) 



Many of these transformations have involved significant allocations of university resources in the form of time, 
money, and financial aid (for example, for doctoral students hired to serve as tutors for students who will be taking 
the MTEL). Deans and department chairs at besieged institutions appear to have been resourceful and inventive 
program advocates who effectively reallocated money to hire consultants, develop test workshops, and measure 
student outcomes. Some institutions with meager financial resources used grant funding to develop test preparatory 
activities; for example, one IHE allocated a portion of federal Eisenhower grant funds to help its students prepare for 
the communication and literacy sections of the test. The costs involved in faculty attending professional development 
sessions relevant to the MTEL and redesigning courses accordingly are important "hidden" costs entailed in the 
teacher testing enterprise. Test preparation has also entailed additional expenses for teacher candidates (at one private 
university, test preparation workshops cost $150). 



At the opposite extreme of testing performance are those colleges and universities with high pass rates that have been 
maintained across all administrations of the teacher test. These IHEs have generally evidenced a minimum level of 
investment in test preparatory activities. Although they may have made modest efforts to adjust curriculum or share 
test-relevant information with other IHEs, they have, essentially, no formal test preparation for their students. 
Orientation sessions, if held at all, tend to be voluntary and focus on only the general test format. Review or study 
materials may be distributed to students but there are no formal assignments. Likewise, there tend to be no 
requirements for passing any components of the test prior to entering the program or prior to student teaching. Six or 
15% of the institutions that responded to our survey fit into this category. 



Institutions with minimal investment give evidence of virtually no discussion between teacher education faculty and 
their arts and sciences colleagues regarding subject matter competence. There appears to be no serious effort to 
identify at-risk students or students who flunked sections of the test. Nor do they feel compelled to maintain data files 
for statistical analysis of performance patterns. This tone of satisfaction with the status quo is captured well by one 
Title II coordinator at a small liberal arts college, who reported, "we have done nothing in the way of gate-keeping or 
special preparation of our students for the MTEL." In a similar vein another coordinator stated that "we feel that our 
curriculum is the preparation for the MTEL." 



IHEs with minimal investments do tend to advise students to split the tests into separate testing sessions and they may 
provide handouts with test-taking advice and materials to serve as general references for self-study. Responsibility for 
test information, record keeping, and interaction with the state and the test contractor tends to fall on the shoulders of 
a single person, typically the department chair. At more than one such institution, however, the secretary for the 
department of education in the college is designated the MTEL coordinator who is responsible for maintaining 
databases and testing materials. 



What is particularly interesting about these institutions is the fact that some of them will be adversely affected when 
the next round of Title II results are released (for the 2000-2001 period). Specifically, some of them will lose their 
ranking in the top Title II reporting quartile. This shift in ranking will happen because many institutions have recently 
implemented policies that require prospective teachers to either pass the tests prior to admission to the teacher 
preparation program or as part of their student teaching component. Thus, there will be an increase in the number of 
programs that claim a one hundred percent pass rate on the test in the next round of Title II reports. Some institutions 
are concerned about this particular public relations aspect of the test. One Title II coordinator said that "We plan no 
special preparation programs, but perhaps we should be thinking of requiring the basic skills test as a teacher 
education prerequisite in order to join those institutions that now are reporting a one hundred percent pass rate." 



o 

ERIC 



Institutions with a moderate level of investment (typically, those below the state cut-score of 80% passing but not so 
low as to have been shocking) generally provide required orientation sessions or workshops where the format of the 
test and test-taking strategies are described by teacher educators or affiliated staff Student performance data are 
gathered for systematic statistical analysis and test performance over time is tracked. At-risk students and those who 
failed a test are identified and tutoring sessions are provided. The administrative support network tends to be 
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elaborate, involving program faculty, the practicum office, and usually a dean. Thirteen or 33% of the institutions or 
our responding institutions are in this category. 

At some colleges and universities with moderate levels of investment, Saturday sessions are held for the two weeks 
immediately prior to the tests. Take-home exercises relevant to these sessions may be required and student advisors 
are encouraged to meet with each student individually before the test is administered. Teacher education faculty and 
affiliated staff explain test registration booklets to students; further, teacher educators design and share information in 
workshops to familiarize students with different types of test questions and to encourage time management strategies. 
Some Massachusetts teacher educators encourage teacher candidates to visit web site locations, such as that of the 
Texas Education Agency (http://www.excet.nesinc.com/excetstudyguid/); this site provides sample test items on the 
ExCET test for Texas teacher candidates, which is in many ways analogous to the MTEL and is also designed by the 
NES. 

Some sections of the tests are required for admission to the teacher preparation program and some programs require 
the students to pass all sections of the test before student teaching. One example of an institution with a moderate 
level of investment is a private university which distributes test preparatory information from the Massachusetts 
Department of Education and refers students to test preparatory workshops at a nearby state college. The university 
advises its students not to take all three sections of the MTEL on the same day and plans to require the teacher 
candidates to pass the communications and literacy sections of the test before they enter student teaching. An 
administrative assistant tracks student test scores and analyzes the results to ascertain any possible consequences for 
the teacher education program. 

Cross-Institutional Collaborations 

Across all three levels of institutional investment, faculty and staff shared some degree of confidence about their 
ability to prepare test-takers for the communication and literacy sections of the test. There was much less certainty 
about the academic content area portions. As one program coordinator confessed, "When it comes to subject matter 
tests, our students are on their own." Expressing similar sentiments, another respondent said, "It is interesting to me 
that with such emphasis being placed on the test results, no person from the Department of Education has offered to 
give us workshops with useful information on exactly how we can prepare the students. Most of the information and 
strategies have been developed through networking with other education program faculty." 



Driven by this need to share information, colleges and universities with teacher preparatory programs in 
Massachusetts have sought forums to discuss the MTEL and to develop appropriate institutional responses. For 
example, Framingham State College convened a Conference on MTEL Test Preparation on 7 January 2002 to assist 
teacher educators in sharing and analyzing information relevant to the test. Likewise, a regional workshop funded by 
the Alliance for Education supported the Colleges of Worcester Consortium in their efforts to better understand test 
preparation practices (Ludlow, 2002). The Association of Independent Colleges and Universities in Massachusetts, 
led by Clare Cotton, has been instrumental in recognizing the need for IHEs to confer about the MTEL and to 
improve communication with National Evaluation Systems and the Massachusetts Department of Education. Finally, 
the Massachusetts Coalition for Teacher Quality and Student Achievement has made the preparation of teachers for 
urban schools a priority and focused on teacher tests at several of its conferences and institutes. 



These are all voluntary efforts that institutions freely engage in at their own discretion. Perhaps not surprisingly, 
besieged institutions have been most visible in these broad-based networks. They report that sharing information 
across institutional lines has helped them to develop programs and to gamer resources from their deans and 
presidents. Institutions with minimal investment generally do not participate in these forums, and those with moderate 
investment participate only intermittently. 



Complicating these facets of preparing for the tests, is the fact that many teacher educators previously had little, and 
in some cases no, contact with their arts and sciences colleagues who teach academic content knowledge. Hence, one 
consequence of the teacher test has been to promote collaboration between teacher education and arts and sciences 
faculty. One problem is that in some areas, such as history, Massachusetts has been unable to establish state 
curriculum frameworks, so even those cannot be referred to as a point of reference. 



One wide-spread approach to the subject matter tests has been for individual academic departments to assume the 
responsibility for providing informal sessions on subject preparation for the tests. For example, at one institution 
faculty in both the English and History Departments offer optional workshops for students. These provide a review of 
sample subject matter questions that are provided by state and test contractor documents. Ironically, faculty who 
wanted to do a good job preparing their students for the tests were frustrated by the wide range of questions that could 
be posed to students. "You would basically have to know the history of the world to ace this exam," one professor at a 
well-regarded private college stated. 

Second Question: Insights from Institutional Data Analyses 

Our second research question addressed analyses of data relevant to the teacher tests. We were particularly interested 
in the extent to which IHEs tried to systematically organize and analyze their test results for the purpose of better 



understanding the strengths and weaknesses of their students and their own teacher preparation programs. Given the 
initial relatively high failure rates, what student and program variables might be useful for understanding students' 
poor performances? If such variables could be identified, then how might IHEs assist future test-takers to prepare for 
the tests? 

The following examples illustrate the kinds of statistical analyses that have been performed as more colleges and 
universities realize the potential benefits to be derived from the test score data. 



• Correlational analyses have revealed positive relationships between the Communication and Literacy Skills 
Tests (CLST) and the subject matter tests and between the CLST and subject tests with SATs and GPAs. 

• Analyses of test-retest effects on the CSLT and subject matter tests have indicated that students who fail the 
first time are likely to pass on the second administration. 

• Longitudinal charts have revealed strengths and weaknesses in sub-areas over time. 

• Ordinary least squares multiple regression models have been successfully constructed for the purpose of 
using GPAs and SATs as predictors of at-risk test-takers. 

• Logistic regression models have been successfully employed with demographic and programmatic variables 
to predict success or failure on the tests. 

• Independent means t-test analyses have not found significant differences in test scores based on gender but 
have found differences based on academic level (undergraduate versus graduate student). 

These analyses were not usually performed for the purpose of hypothesis testing. They were usually conducted as 
exploratory analyses attempting to reveal any patterns or relationships among test results, student characteristics, and 
program structures that would shed light on why students either passed or failed and, more importantly, how student 
preparation for the tests could be strengthened. 



There have been results, however, that were significant that have been shared between teacher preparatory programs 
in different colleges and universities. Some of the most significant findings are of potentially tremendous value to 
teacher candidates and teacher educators. For example, many students who took all three tests on the same day 
performed worse than those who took the third component of the tests (the subject matter test) on a separate day. 
Accordingly, most institutions now advise students not to take all three sections at once. Another example is that 
graduate students have outperformed undergraduate students, so some institutions have focused their test preparatory 
activities on undergraduates. Students with low SAT scores have performed worse than those with high scores, and 
students with low GPAs have performed worse than those with high GPAs. Some colleges and universities have used 
this data to identify at-risk students and to develop appropriate interventions for them (Note 3). 

One besieged institution has defined students at-risk of failing if they have an SAT verbal score below 420, did poorly 
in basic college writing or introduction to education courses, have English as a second language, or have a learning 
disability. These students are required by the university to take test preparatory workshops that range from fifteen to 
twenty-four hours in duration. The workshops last for several weeks, with multiple opportunities for workshop leaders 
to assess student progress over time. 



One major problem that every IHE in Massachusetts has faced in performing these analyses is that the test scores are 
not available in electronic format. (Note 4) Thus, any statistical analysis first requires hand processing of the paper 
records from the testing contractor. This, in turn, forces each IHE which seeks to assist its teacher candidates to 
develop its own approach to building databases and performing analyses. It has been impossible thus far to perform 
any cross-institutional aggregation and analysis of data (Note 5). In some cases these data management problems have 
prevented institutions from analyzing their own data. As one Title 11 coordinator noted: "It takes too much time now 
for me to enter all the data and as soon as we train a graduate student to do data entry the student graduates. 
Unfortunately, we have not had a statistician or faculty member that has taken an interest." 

Third Question: Results of Test Preparatory Activities 

Our third research question concerned the efficacy of test preparation efforts in relation to improved test scores. We 
know little to date about the impact of the new teacher test preparation approaches, in spite of the national wave of 
innovations in this area in recent years. The impact of a teacher test preparation program in Arizona was examined by 
Fierros (2002) but his effort was specific to a single institution and explored candidates’ sentiments about the program 
rather than their actual test score results. Consistent with Fierros' positive findings, we also received many favorable 
comments about the test preparatory programs in Massachusetts. For example, one Title II program coordinator said 
that "students feel more confident and tend to do better on the first try if they attend the one-credit test prep course . . . 
students benefited greatly from the one-on-one tutoring ... the students really feel it helps." Another coordinator 
commented, "Everyone benefits from some MTEL test preparation. It is very helpful for even the most skilled test 
takers to attend a two to three hour orientation/test readiness course." 

Although anecdotal evidence is useful, it is also possible to establish a statistical relationship between increased test 
preparation and test performance on the teacher tests. Starting in April 1998, the Massachusetts Department of 
Education began releasing institutional test results after each administration of the teacher test. This practice 
continued through June 1999, at which point the Department ceased making public reports on the data (with the 



exception of the Title II test results in April 2001). 



Recall that we disaggregated our teacher preparatory institutions into three categories: those with a minimal 
investment in improving scores, those with a moderate investment; and those which are heavily invested. The latter 
group we labeled besieged institutions because failure to increase pass rates above 80% by 2004 could result in the 
Department of Education closing their teacher education programs. 



MTEL Pass Rates: 

First six tests and first Title II report period 




Test Date (2001 refers to the 1999-2000 period) 



The chart represents the summary pass rate for each of our resource investment categories across each of the MTEL 
testing dates. "Summary pass rate" is defined as "the proportion of program completers who passed all tests they took 
for their areas of specialization among those who took one or more tests in their specialization areas" (United States 
Department of Education, 2000). The bold horizontal line at 80% pass represents the Massachusetts Department of 
Education criterion for institutional approval for the continuation of a teacher preparatory program (Massachusetts 
Department of Education, 1998). The last test date (2001) refers to the September 1, 1999-August 31, 2000 Title II 
reporting period. Those results were submitted to the Department in April 2001 . 



The minimally invested institutions started off with high pass rates and have maintained a relatively constant high 
level of success on the tests. In essence, they were not threatened by the test and did not need to exert any additional 
efforts to meet the state standards, although several of them have recently made some efforts in this direction. 

In contrast to their situation, institutions which are moderately or heavily invested in improving their pass rates have 
shown a steady improvement over time. The moderate category institutions show a sharp rise ending the 1999 
academic year and, as a group, exceeded the 80% threshold for the Title II reporting period. Besieged institutions 
started off about fifteen percentage points below the moderate group and have stayed roughly at a fifteen to twenty 
point difference across time. For the first Title II reporting period the besieged institutions were below the 80% 
threshold. Not only have they have dramatically narrowed the performance gap (Note 6) but, based on their trajectory, 
we anticipate that they will meet the state standard as a group when the next round of test score results are released. 
(Note 7) 

There is no question that the improvement of test scores is correlated with the rise of test preparatory activities in the 
besieged colleges and universities. The extent to which the rise in test scores was caused by those activities is, 
however, unclear because there are confounding variables for which we have no controls. For example, the extent to 
which teacher education curricula have changed to conform with MTEL content is unknown. Another variable is the 
extent to which admission selectivity (e.g. high school GPA and SAT scores) has become more rigorous. 



We do not believe that these confounding variables should be given much weight. Teacher educators had little 
information about the teacher test during the first administrations and could only rely on test-takers to learn about the 
nature of the test. Anita Page, the director of the early childhood and elementary programs at Mount Holyoke College, 
asked "Well, what is it that 1 have to improve?," reflecting a widespread sentiment of concern about the lack of clarity 
in Department of Education guidelines (Tantraphol, 2002). Regarding student selectivity, it seems implausible that 
any increase in teacher preparatory program admission criteria could affect test results, in the aggregate, in such a 
narrow span of time. 



Conclusion 



As we have seen, besieged institutions responded rapidly with innovative strategies to enhance the content knowledge 
of prospective teachers as well as their writing and reading skills. In addition, some institutions changed their 
admission criteria, student teaching requirements, and program completer definitions to include passing the MTEL. 
Subsequently, test takers' scores in the besieged institutions improved dramatically in the years following the initial 
administration of the teacher test. At present, it is impossible to disentangle to what extent specific factors led to the 
rise in test scores. 



Based on the results described here, the faculty, administrators, staff, and students at many institutions may feel 
pleased that their strategies improved teacher test results — and, by implication, the skills and competencies required 
of all teachers. Furthermore, advocates of teacher testing might claim that state and federal teacher testing policies are 
working effectively to upgrade the teaching profession. 



We, however, wish to advance several caveats because we share with critics of teacher testing concerns about how the 
tests are being used and the manner in which they are transforming teacher education. First, many of the besieged 
institutions recently changed the time of their testing so that students take sections of the teacher test before even 
being admitted to a teacher education program. In these cases, the institutions are guaranteeing a high pass rate from 
their program completers. Second, we do not know if valuable facets of teacher education have been sacrificed in the 
effort to improve test scores. Cochran-Smith and Dudley-Marling (2001) found that significant institutional resources 
were devoted to test preparatory activities and some of these resource allocations detracted from socially critical parts 
of a university's mission, such as recruiting students of color into teaching. Finally, the test may screen promising 
teachers out of the profession who could be quite effective in classes yet do not perform well on standardized tests. 

Teacher testing — virtually unknown two decades ago — has now become ubiquitous in the United States. As a result 
of this far-reaching transformation, teacher educators are now able to use test data to review, analyze and strengthen 
their programs. When combined with other innovations in the field of teacher education — such as portfolio 
assessment, exhibitions, and teaching demonstrations — teacher testing can play an important role as one strand of 
holistic assessment. Professional approval programs, such as the National Council for the Accreditation of Teacher 
Education (NCATE), require schools, colleges, and departments of education to document their self-study efforts, and 
teacher test results provide additional information on teacher candidates' academic competencies. 




Critics, however, have raised serious concerns about unintended consequences of teacher testing. These critics will 
not, and should not, be silenced simply because there are positive trends in teacher test results. Even as we applaud 
the progress made by besieged institutions in improving test score results, we must continue to inquire into deleterious 
results of testing that detract from essential components of teacher preparation. 

Notes 

1 . Anna Maria College, Becker College, Berklee College of Music, Boston College, Boston University, 

Brandeis University, Bridgewater State College, Cambridge College, Clark University, College of the Holy 
Cross, Curry College, Eastern Nazarene College, Elms College, Emerson College, Emmanuel College, 
Endicott College, Fitchburg State College, Framingham State College, Gordon College, Harvard Graduate 
School of Education, Lesley College, Massachusetts College of Art, Massachusetts College of Liberal Arts, 
Merrimack College, Montserrat College of Art, Northeastern University, Salem State College, Simmons 
College, Smith College, Springfield College, Stonehill College, Suffolk University, Tufts University, 
University of Massachusetts/Amherst, University of Massachusetts/Boston, University of 
Massachusetts/Lowell, Wheaton College, Wheelock College, and Worcester State College. 

2. Some institutions report that obtaining useful practical and technical information about the MTEL is still 
problematic. In fact, no technical reports have been released "following the use of each form of the tests" and 
no technical advisory committee has been formed to "meet up to four times annually to review the test items, 
test administration, scoring procedures, and score setting for validity and reliability", even though both 
actions have been called for in the MTEL contracts. 

3. The opportunity for IHEs to conduct these valuable statistical analyses upon their own candidate test score 
results has just recently become more restrictive. In an April 18, 2002 letter to Judith Gill, Chancellor, Board 
of Higher Education and Clare Cotton, Executive Director, Association of Independent Colleges and 
Universities in Massachusetts, Commissioner of Education David Driscoll stated that candidates will now be 
required to give explicit "consent before his or her institution is sent any information on individual subarea 
performance." This requirement is imposed even though Driscoll acknowledged that language in the current 
registration booklet is "essentially identical in nine of the states served by the National Evaluation Systems." 
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This additional release will inevitably result in fewer complete data records for teacher candidates and their 
institutions. 

4. Each contract for the MTEL has stipulated that the test developer (National Evaluation Systems) provide the 
test scores in electronic format to IHEs (just as the NES presently does for Title H reporting purposes). The 
contract specifies that 'The Contractor will... electronically transferring official scores to the institutions after 
each administration." After four years of testing, however, the NES still does not provide the data on an 
electronic medium suitable for processing by standard software like SPSS or EXCEL, 

5. Commissioner Driscoll stated that "There are two reasons for my prohibition on cross-institutional pooling of 
data. First, we wish to protect the individual’s identification. Second, we wish to lessen misuse of our 
licensure tests." {ibid). The first point can be easily addressed on the data records using standard procedures 
such as codes and pseudonyms to protect individuals’ confidentiality. Regarding the second point, we do not 
know what kinds of "misuse" the Commissioner has in mind. The practical consequence of this prohibition is 
to further reduce the capability of IHEs to serve their students and the interests of the general public by 
improving teacher candidates’ test scores. The political impact of this prohibition is that independent analysts 
are prevented from conducting the sorts of rigorous validity and differential impact analyses that could assist 
teacher candidates. 

6. It is important to note that the dramatic increase from June 1999 to the 2001 Title II results in the last column 
is partially attributable to different definitions of test-takers. Prior to Title II, test-takers included anyone who 
claimed affiliation with a college or university. There were no controls on the population of test-takers, i.e. 
anyone who took the test and claimed an institutional affiliation was counted in institutional results. For Title 
II purposes, however, only "program completers" are test-takers. According to Title II guidelines as provided 
by the Massachusetts Department of Education, program completers are only those individuals who took the 
teacher test and met all other institutional program requirements for graduation and certification. This 
distinction has led to the apparent paradox that presently exists when test administration results are released. 
That is, "On the February 23, 2002 administration, a total of 58% of 5,225 first time test takers passed all 
three parts of the test" (Massachusetts Department of Education, 2002) yet the statewide pass for Title II 
(1999-2000) was 81%. 

7. Congress requires IHEs to send their Title II test results to their respective state agencies by April 7 of each 
year. The Massachusetts results for 1999-2000 were subsequently posted on most institutional web sites, in 
accordance with the requirement that the results be available to the public. This year, however, most IHEs 
have not posted their results on their respective web sites as of 22 June 2002 — and we checked 58 separate 
IHE sites. Furthermore, the Department of Education declined to provide those results for this article. Thus, 
even though in theory the latest Title II results are accessible by the public we could not in point of fact obtain 
and use them. 
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